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PREFACE 


This book and its accompanying software are my attempt to bring together 
in one package the basic theory, methods, and algorithms that are used to ana- 
lyze univariate and bivariate linear time series in both the time and frequency 
domains. I have tried to be complete and rigorous in my treatment of each of 
the three areas, but at the same time I have tried to organize the material in 
such a way that makes it possible for readers particularly interested in one area 
to find their way around in the package without too much distraction. My main 
aim has been to provide a set of tools that allows a reader to study the various 
parts of the subject. I have called the result a laboratory as its emphasis is 
not primarily on providing “canned” programs for doing traditional analyses, 
but rather on allowing the interested reader to study the theory and methods 
involved in the subject. A set of routines (called macros) is in fact included 
that allows TIMESLAB to be used as a canned program, but the reader is 
encouraged to study the subject in more detail. 

TIMESLAB grew out of a series of lecture notes and computer programs 
that I developed over ten years in a course in time series analysis for students 
having varied interests and backgrounds in mathematics and statistics. These 
students have included (1) business students primarily interested in forecast- 
ing, (2) engineers wanting to learn spectral analysis, (3) students in statistics 
wanting to learn general theory and methods, and (4) students whose primary 
interest was in the computational aspects of the subject. All of these stu- 
dents have had in common a familiarity with the basic ideas of probability 
and distributions, a knowledge of regression analysis from a matrix point of 
view, and the desire to learn about time series analysis. The software that I 
developed originally took the form of both a library of Fortran subroutines and 
a collection of batch style main programs. Both of these were difficult to use 
and expensive to run. As personal computers became available in the early 
1980s, it became clear that they made possible the creation and widespread 
use of command-driven interactive graphics style programs for studying a wide 
variety of scientific subjects, including time series analysis. 


vii 
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The Program 


The TIMESLAB program consists of approximately 150 “commands,” 
each consisting of an operation such as reading, plotting, transforming, or 
forecasting data. Several graphics commands are available (all of the figures 
included in the book were created using the program), a text editor is built 
in, and it is possible to issue operating system commands while inside the pro- 
gram. The user tells the program what to do either by entering commands 
one at a time from the keyboard or by issuing a command telling TIMESLAB 
to execute a series of commands contained in a file. Enough conditional and 
unconditional branching commands are provided for use in these (macro) files 
so that TIMESLAB is actually a time series analysis programming language. 
Thus the commands can be thought of as a library of subroutines, while the 
macros can be thought of as main programs. The users can use the 80 macros 
discussed in the book and provided with the program, or else create their own 
macros for performing analyses not covered by the supplied macros. An ap- 
pendix (Appendix B) is included that guides the user through the process of 
creating macros, and the code of each of the supplied macros is listed in the 
book to provide models for the writing of others. 

In addition to commands directly associated with time series analysis, 
there are several general-purpose commands, including matrix operations, eval- 
uating the pdf, cdf, and quantile functions of the Z, t, y?, and F distributions, 
a general nonparametric probability density estimation procedure, and several 
commands for analyzing polynomials. This has led to its use in a variety of 
non-time series analysis applications. 


The Textbook 


I have tried to provide a balance of theory, methods, and algorithms in 
the book. The theory is stated rigorously but in a manner in which the reader 
interested only in applications can skip over the details. The basic results for 
each major topic are gathered into one (often lengthy) theorem. At the end 
of many of the theorems I have included a section describing the implications 
of the theorem. This approach makes it possible for readers to go directly to 
the material of interest to them without having to search through a series of 
definitions, theorems, and proofs. Some proofs are included if they are not 
extremely difficult and not widely available elsewhere. If TIMESLAB were to 
be used in a course principally devoted to theory, the instructor would have 
to supplement the proofs included in the book. References to places where 
proofs can be found are included in the cases where they are not given. An 
index to the theorems is included in Appendix E, while Appendix A contains a 
discussion of the more difficult mathematical and statistical background needed 
for those interested in the theory. In the one-semester course described above, 
I have emphasized understanding of these implications more than the rigorous 
statements themselves. 
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I have taken the same approach with the algorithms that are used in the 
TIMESLAB program. For example, the fast Fourier transform is described 
briefly in Chapter 1 of the book and more extensively in Appendix A. Many of 
the algorithms for univariate time series are carefully explained, but usually in 
separate sections. For example, Section 2.6 is devoted to a description of many 
of the basic algorithms for univariate time series. 

For those readers primarily interested in the application of time series 
methods, several of the macros can be used as models for similar analyses. 
Appendix E contains a list of all of the examples and macros that are included 
in the book. 

The basic point of view taken by the book is that much is to be gained 
both in theory and in applications by combining what are called the time 
domain and frequency domain approaches to time series analysis. The emphasis 
is on the use of the correlogram, partial correlogram, and spectral density 
function for both model determination and as diagnostic tools. Many of the 
ideas of time series analysis are presented in analogy with ideas from regression 
analysis. A section is included in Appendix A reviewing these ideas. Thus the 
usual modeling strategy of regression analysis is employed with a wide range 
of models and diagnostics, including those from the frequency domain, being 
easily accessible in the TIMESLAB program. 

The book is organized into four chapters, a mathematical and statistical 
appendix, two appendices describing the TIMESLAB program, an appendix 
containing time series data sets, and finally an appendix that is an index to 
the commands, theorems, macros, examples, and figures that are in the book. 
The first three chapters are concerned with univariate time series, while the 
fourth discusses bivariate series. In Chapter 1 the ideas of the time and fre- 
quency domains are introduced in terms of describing and classifying time series 
data sets. The sample correlogram, partial correlogram, and spectral density 
are introduced as tools for making an exploratory analysis of data. This ex- 
ploration is presented by determining whether transformations of the data are 
required. Traditional data tranformations are then discussed and some simple 
forecasting methods presented. Chapter 1 concludes with a discussion of some 
basic methods for describing the distribution of a data set. This is useful for 
investigating assumptions made about data and also as a tool to be used later 
for some numerical experiments illustrating theory. 

Chapter 2 presents the basic probability theory of covariance stationary 
time series, with a discussion of the traditional models used in time series, and 
the basic theory of spectral analysis and minimum mean square error predic- 
tion. Chapter 3 consists of a description of a variety of statistical inferences that 
can be made from data. The sampling properties of the descriptive statistics 
introduced in Chapter 1 are given, two tests for white noise are described, and 
traditional nonparametric spectral estimation is presented in some detail. In 
-Sections 3.4 and 3.5, the modern methods of parameter estimation and model 
order determination are described. In Section 3.6, the popular Box-Jenkins 
modeling and forecasting method is described, while Section 3.7 discusses other 
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such model-based procedures, including regression with autocorrelated errors 
and iterated autoregression modeling. After all of these modeling techniques 
are developed in previous sections, Sections 3.8 and 3.9 describe their use for 
spectral density estimation and the search for periodicities, respectively. 

The analysis of bivariate time series is described in Chapter 4 with the 
emphasis being on the ideas new to that situation. These ideas include cross- 
spectral analysis and transfer function modeling. 

At the end of each of Chapters 1~4, there are a series of examples that 
either are used in the chapter or supplement the ideas of the chapter. There 
are also two sets of problems at the end of each chapter: one set requiring the 
reader to use the program to illustrate a theoretical or methodological point, 
and the other set not requiring the program. 


Suggestions for Using TIMESLAB 


TIMESLAB has been designed to be used in a variety of types of courses. 
For a two-semester sequence for students with a good background in math- 
ematics and statistics, the entire book could be covered, with Appendix A, 
Chapters 1 and 2, and the first six sections of Chapter 3 covered in the first 
semester and the rest in the second semester. The sections on computing as- 
pects (Sections 2.6 and A.1 and A.3) could be optional. I would assign about 
one third to one half of the computational and theoretical problems from each 
chapter. 

A one-semester course in applications of univariate time series analysis 
could consist of Sections 1-6 of Chapter 1, a rapid treatment of Sections 2.1, 
2.2, and 2.5, and then an introduction to each of the topics in Chapter 3. 
Approximately one fourth of the course would be devoted to each of the first 
two chapters and half to the third. The emphasis would be on the implication 
of the use of the theorems, and the supplied macros and examples would be 
used to make it possible to cover all of the material. 
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CHAPTER 1 


Describing, Transforming, and 
Forecasting Univariate Time Series 


Almost every area of scientific inquiry is concerned with data collected 
over time, that is, with time series. Figure 1.1 contains the graph of ten such 
time series. A listing of all of the time series analyzed in this book is included 
in Appendix D, and the diskettes that accompany the book include a file for 
each data set. 

We have chosen these series to begin our discussion because they illustrate 
some of the different types of series with which we will be concerned and the 
types of scientific questions that one can answer using time series analysis. The 
aim of this chapter is to introduce some of the basic descriptive techniques used 
in time series analysis and to aquaint the reader with the TIMESLAB software 
that accompanies the book. We will also begin our discussion of forecasting 
time series. 


1.1. Time Series Data Types 


Time series can be classified in many ways, including by the following four 
characteristics: 


1. The dimension of the index set T’ over which observations are made. The 
set T can be one-dimensional or multidimensional. Daily births of fe- 
males in California (Series I) have a one-dimensional index set, while wheat 
yields recorded over a regular grid of positions in a large field have a two- 
dimensional index set. This second example also illustrates that the index 
set need not literally be “time” but can also be “position.” 


2. Whether the index set is continuous or discrete. Thus an analog EEG 
record from one probe is a continuous one-dimensional series, while daily 


birth data are a discrete one-dimensional series. 


3. How many variables are recorded at each element of the index set. Uni- 


2 DESCRIBING AND TRANSFORMING SERIES CH. 1 


Series I: Daily California Births 


80. 
68. 
%. 
“4. 
32. 
28 


14, 148, 222, 2%. «378, 


Series I]: Beveridge Wheat Price Index 


1360. 1648. 1720. 1860. 1880. 


Series III: Normal White Moise 


“4. 6. 


66. «88, 118, 


Series ¥: Artificial Series 


20. 6. 6M, 88, 108, 


Series VI: Monthly Airline Passengers 


7. 
38. 
420, 
280. 
1%. 

@. 


120. 158. 


_ denies VII: Random Walk Series 


» @, 80, 120, 160. 208. 


Series VIII: Lake Erie Levels 


1931, 1941, 1951. 1961. 1971. 


2. Series IX: Sun of Four Cosines 


12. 
4, 
4, 
“12. 
20 
3. 668, M128, 


_ Series X: Bivariate Series 


68. 120, 180, 240, 300 


Figure 1.1. Some Examples of Time Series. 
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variate (also called scalar) time series have one variable at each time, 
while multivariate time series have a vector of variables measured at each 
time. For example, a series consisting of monthly interest rates and gross 
national product is a one-dimensional bivariate time series. Series X is an- 
other such example, as the input and output values of a gas in a chemical 
process are recorded at each time. 


4. Whether the variable or variables at each time are themselves discrete or 
continuous. Most series are continuous; that is, each variable can take on 
a continuum of values. An example of a discrete valued series is a binary 
time series (see Example 1.7), one in which an observation can take on 
only one of two values. 


To summarize then, time series can be classified as follows: 
1. One or multidimensional (dimension of index set). 
2. Discrete or continuous time (nature of index set). 
3. Univariate (scalar) or multivariate (number of variables). 
4. Discrete or continuous space (nature of variables). 


Note that discrete-time time series can also be either equally spaced or 
irregularly spaced in time. In this book we will be almost exclusively concerned 
with time series that are one dimensional, equally spaced, discrete in time but 
continuous in space, and either univariate (Chapters 1-3) or bivariate (Chapter 
4). 


1.2. Time Series Memory Types 


The basic property of time series analysis is that it is concerned with 
repeated measurements on the same phenomenon at different times or places. 
Because of this, the analyst must take into account the correlation between 
successive observations. This is in marked contrast with the data analyzed in 
elementary statistics courses where one assumes that the data are made up 
of independent and identically distributed observations obtained by randomly 
sampling some population or populations. The presence of correlation makes 
the analysis of time series data and the interpretation of the results much more 
difficult than in the independent case. 

In this section we classify time series into three broad classes according to 
what we will call their memory type: 


1. Purely Random Series. This type of series shows no patterns over 
time. Series III is an example of such a series. It was generated by TIMESLAB 
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using a random number generator so that it would be indistinguishable from a 
random sample from a standard normal distribution. Series IV, on the other 
hand, is a real data set (monthly total rainfall), but appears quite similar 
to Series III. We will see later that such data are aptly named white noise. 
Purely random series are also called no-memory series, as one characterization 
of statistical independence is that an observation at one time has no memory 
of the observations at any other time. 


2. Long-Memory Series. This type of series is the opposite extreme of 
white noise; that is, a plot of the data looks to be almost that of a deterministic 
function of time. Series V and VI illustrate this type. The first was artificially 
generated as values lying on a cosine curve that goes through ten cycles with 
small random numbers added to each point. Series VI is a real economic time 
series (monthly total international airline passengers for 12 years). These two 
series have in common that both could be almost perfectly extrapolated far 
into the future unless something were to happen to the mechanism generating 
the data. This is the origin of the term “long memory.” The dependence on the 
past does not die away quickly. Note that many of the time series in business 
and economics are long memory. 


3. Short-Memory Series. This type lies between white noise and long 
memory, occurs often in the physical and engineering sciences, and comprises 
the bulk of the time series that can be most usefully analyzed by the sophis- 
ticated methods of time series analysis that we will study. Series VIII and IX 
appear to be short-memory series; clearly observations close together in time 
are more similar than those far apart in time, but there are no apparent de- 
terministic patterns in the data (although upon closer inspection you might 
be able to tell that Series IX is actually the sum of four cosine curves having 
squared amplitudes 10, 13, 37, and 65 and going through 4, 12, 24, and 48 
cycles, see Example 1.4). In a short-memory series the predictability of the 
observations at one place in time from past observations appears to die out 
quickly as time goes on. 


Time series are usually classified as stationary or nonstationary. We study 


these concepts in detail in Chapter 2, but for now we will think of nonstationary 
as long memory and stationary as white noise or short memory. 


The WN Command 
The TIMESLAB command 
x=WN(seed,n,dist) 


will generate a sample realization of length n from a white noise series (using 
the random number generator seed entered in the real scalar argument seed) 
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Table 1.1. Distributions in the WN Command 


dist Distribution Mean Variance 
ee ty WRIA 0 1 
2 U(0,1): U 1/2 1/12 
3 Exponential: —log(1 — U) 1 1 
4 Logistic: log(U/1 — U) 0 n? /3 = 3.28987 
5 Cauchy: tan(m(U — $)) foe) fone) 
6 Extreme Value: log(—log(1—U)) -y=-—.5772 7?/6= 1.6449 
7  Lognormal: e” e/2 — 1.6487 e? —e = 4.67077 
8 Double Exponential 0 2 
ee sD 
—log(2(1—U)), U>.5 


for any of the distributions in Table 1.1. 
1.3. Aims of Time Series Analysis 


The aims of time series analysis can be roughly grouped into four cate- 
gories: descriptive, inferential, predictive, and control. Time series analysis 
is extremely graphical in nature because of the lack of independence among 
the quantities that it investigates. In the next section we will consider three 
graphs that can be used for trying to classify a time series into long, short, or 
no memory. 


In Chapter 3 we will consider the inferential part of univariate time series 
analysis. Essentially we seek to determine from one time series data set. what 
other data could have been observed or are expected to be observed in the 
future. To do this we must be able to assume that the random mechanism 
generating the observed data follows some kind of model. These models are 
introduced in Chapter 2. 


The third aim is to use the correlation in time series to predict the future. 
In Chapter 2 we introduce the basic theory of prediction, and in Chapter 3 we 
give several examples of prediction methods. Given the ability to predict the 
future of a time series, one naturally begins to think of controlling that future. 
Thus one can begin adjusting the values of time series so that the future values 
of the series are in line with what is desired. 
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1.4. Basic Descriptive Statistics 


The first aim of any statistical procedure is to give a succinct description 
of the data being analyzed, both graphically and numerically. In time series 
analysis there are three basic graphical techniques for describing data: the 
correlogram, the partial correlogram, and the periodogram. In this section we 
introduce each of these quantities in turn and illustrate how they are used to 
describe data z(1),...,2(n). 


1.4.1. The Sample Correlogram 


The distinguishing characteristic of a time series is that it can exhibit serial 
correlation, that is, correlation over time. For example, Figure 1.2 contains 
scatterplots of x(t) versus z(t—1) for t = 2,...,n for Series I and II. Note that 
for Series I there appears to be little correlation in the plot, while in Series 
II there appears to be high positive correlation. The macro that was used to 
produce Figure 1.2 is given in Example 1.3. 

In elementary statistics we use the sample correlation coefficient 


dover (2s — 2) (4s — 9) 


p= = : 
Dini (25 ion z)? ia (Yi coal y)? 
to measure correlation in a set of bivariate data (71,41), .--, (fn, Yn). Ina 
time series z(1),...,2(n), we usually want to measure the correlation of the 


data with themselves except “lagged” a certain number of time units. Thus for 
a lag v, we have n — v pairs of 2’s that are separated by v time units, namely 
the pairs 


(x(1),2(1+ v)), (z(2),2(2+0)),...,(z(n—v), 2(n)). 


One way to measure correlation in these pairs would be to apply the above 
formula for the sample correlation coefficient. However, the traditional way 
to measure serial correlation in time series is by the sample autocorrelation 
coefficient as given in the following definition. In Section 3.1.2 we discuss why 
this definition is used. 


Definition. Let r(1),...,2(n) be a univariate time series data set. The sam- 
ple autocorrelation coefficient of lag v is given by 


A(v) = Pil (2() a z) bts ake) pa a)» v<n, 
Dt=1 (2(t) — 2) 
where Z is the sample mean of z(1),...,2(n). A plot of p(v) versus v for 
v = 0,1,...,M for some maximum lag M is called the correlogram of the 


data. 


SEC 1.4 BASIC DESCRIPTIVE STATISTICS Ce 
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“Goo 80.00 160.00 208.00 320.00 00.08 
Figure 1.2. Scatterplot of z(t) versus z(t — 1) for Series I and II. 


Note that (0) = 1 and that the largest value of v that can be considered 
is n — 1, since if v > n, there are no pairs of z’s separated by v time units. 
Note further that we will define 6(—v) = 6(v) and thus will sometimes write 


roll (2(t) — 2) (x(t + |v) — 2) 
we (z(t) = z)’ 


Suppose that we are considering a bivariate data set; that is, at each time we 
observe 
z(t) 
z(t) = : Ph ae, Ty 
y(t) 


Then we have the following definition. 


A(v) = » ul <n. 


8 DESCRIBING AND TRANSFORMING SERIES CH. 


Definition. The cross-correlation coefficient of lag v is given by 


tot (2(t) — £) (y(t + v) — 9) 
poy(v) = 2 Nites (2(t) — 2)? De (ol) — 9)" 


pyz(—), v=-—(n-1),...,0. 


pa i oer 


Note that fry is not symmetric about v = 0. A plot of pry(v) versus v for 
v= —M,...,M is called the cross-correlogram of z and y. 

Correlograms are used for three basic purposes. To illustrate these pur- 
poses, consider Figure 1.3 where we have given the correlograms for the ten 
series introduced earlier. A large value of f(v) is indicative of a possible pe- 
riodicity in the data of v time units. For example, the correlogram of a pure 
cosine curve is also sinusoidal (see Problem T1.6). Notice that Series V and 
VI exhibit this behavior. 

A correlogram that does (does not) decay rapidly to zero and then stay 
there as v increases means that the series is short (long) memory. Note that 
the wheat price index, the artificial series, the airline data, and the random 
walk series appear to be long memory by inspection of the correlogram. 

A large cross-correlation of lag v indicates that x and y may be very similar 
except that they are out of phase; that is, one of the series is leading the other. 
This would appear to be the case in the bivariate series that we have been 
considering. 


The CORR and CORR2 Commands 


The TIMESLAB command CORR can be used to calculate the correlogram 
of a univariate time series, while CORR2 finds auto- and cross-correlations for a 
bivariate series. Thus the following sequences of commands will produce a plot 
of the correlogram of a series x and the cross-correlogram for series x and y: 


;Correlogram of x: 


rho=CORR(x,n,M,0,1,R0,per) 
PLOT(rho,M,0,M,—1,1) 


;Cross correlogram of x and jy: 


CORR2(x,y,n,M,1,Rx0,Ry0,rhoxy0,rhox, rhoy, rhoxy, rhoyx) 
rr=REVERSE(rhoyx,M) 

rho=<rr,rhoxy0,rhoxy> 

MM=—M 

ind=LINE(M,0,1) 

indi=—ind 

ind1=REVERSE(indi,M) 
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Series I]: Beveridge Wheat Price Index 
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Figure 1.3. The Correlograms for Series I-X. 
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ind=<ind1,0,ind> 

m2p1i=2*M 

m2pi=m2piti 
LABEL(rho)=’Cross-Correlogram’ 
PLOT(ind,rho,m2pi,mm,M,—1,1) 


We will discuss CORR and CORR2 in further detail in this chapter. 
1.4.2. The Sample Partial Correlogram 


As in elementary statistics, high correlation does not imply causality. We 
can introduce partial and multiple correlations in an attempt to describe fur- 
ther the correlation in certain sets of variables. In elementary statistics, if we 
have random samples from populations Y, Z, and Xi,...,Xp, we can find 
the residuals ey and ez of regressing Y on the X’s and Z on the X’s and 
then find the usual sample correlation coefficient between ey (1),...,ey(n) and 
ez(1),...,ez(n). The result is what is called the sample partial correlation co- 
efficient between Y and Z given X;,..., Xp, and is often denoted pyzjx,...x,- 
The sample multiple correlation coefficient Ry, ges se is the proportion of vari- 
ability in Y that is ‘explained’ by its linear relationship with the X’s (see 
Section A.5 for more detail about the multiple correlation coefficient). The 
analogous quantities in univariate time series analysis are the sample partial 
autocorrelation coefficients and residual variances. The sample partial auto- 
correlation coefficient of lag v is the correlation between x(t) and z(t+v) after 
having removed the common linear effect of the data in between (the lag one 
partial is just the usual lag one sample autocorrelation). We denote the lag v 
partial autocorrelation by 6,. In Section 3.4.4 we discuss in more detail the 
exact definition of by. 


The PARCORR Command 
The command 
part=PARCORR(x,n,ML[,rvar]) 


will calculate the first M sample partial autocorrelations (returned in the array 
part) and optionally the corresponding sample residual variances ¢?,..., 6, 
in the array rvar where G2 is the sample variance of the residuals of having 
regressed x(t) on the previous v z’s. Thus if we divide rvar by the sample vari- 
ance of the z’s and then subtract the result from one, we obtain the time series 
analog of a multiple correlation coefficient. We will usually display what we 
call the standardized residual variances, that is, the residual variances divided 
by the sample variance. These values will always be between zero and one, 
and a small (large) value indicates that x(t) is (is not) very predictable from 


its past, thus indicating that the data are long (short) memory. A useful rule 
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of thumb is that data are long memory if their standardized residual variance 
sequence becomes less than 8/n for some lags. 


In Figure 1.4 we display the partial correlogram and standardized residual 
variances for each of the univariate series that we have been considering. Since 
the two graphs for each series are on scales of (—1,1) and (0,1), respectively, we 
can put them both on the same axes using the PLOT2 command with the partials 
indicated by a solid curve and the standardized residual variances indicated by 
the broken curve. We can use the partial autocorrelations in the same way that 
usual partial correlations are used. In Series VII, for example, the correlogram 
takes some time to decay to zero, while the partial correlogram is large for lag 
one and then small henceforth. This indicates that the correlation between 
z(t) and z(t + 2) is due only to the common relationship of z(t) and z(t + 2) 
to z(t + 1). 


1.4.3. The Periodogram and Sample Spectral Density 


A recurring theme in science is the study of deterministic phenomena that 
have been observed over time (music, speech, radio signals, etc.) via harmonic 
analysis, that is, by decomposing a time record into the sum of sinusoids of 
various frequencies and amplitudes. Harmonic analysis is also important in the 
study of random phenomena observed over time. We will study the theoretical 
aspects of this analysis in Chapter 2 (see also Section A.2), but in this section 
we will motivate the theory by looking at what is called the periodogram of a 
data set r(1),...,2(n). 


The basic idea of harmonic analysis is that one can find a unique set of 
sinusoids (a cosine plus sine of the same frequency) that when added together 
reconstruct the mathematical object being studied. Thus one can study a 
complicated object by breaking it up into these simple “frequency components” 
and studying them separately. Further, in a certain sense, conclusions made 
about one sinusoid are independent of conclusions made about others. 


A similar idea in statistics is that of orthogonal contrasts in the analysis of 
variance, wherein the results of applying different treatments to experimental 
units can be decomposed into the sum of linear, quadratic, and higher-order 
effects that are orthogonal, or statistically independent. 


The mathematical objects that can be studied by harmonic analysis vary 
widely. Examples include sequences of finitely many numbers, infinitely long 
sequences, functions that are periodic, and general continuous or even discon- 
tinuous functions. No matter what object is being studied, the results are 
always of the form of a decomposition into independent sinusoids that can be 
studied separately. 
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Figure 1.4. The Partial Autocorrelations (Solid Curve) and Standardized 
Residual Variances (Dotted Curve) for Series I-IX. 
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Definition. A sinusoid g,(x) of period k (or frequency 1/k) is the function 


gx(2) = acos “= + bsin “2, z E (—00, 00). 


Since cos and sin have period 27, we have that g,(2+kl) = g,(z), for any 
integer 1. We can also write (see Problem T1.2) 


Getty = C cos" — 9), 


where C = Va? + b? and ¢ = arctan(b/a) are called the amplitude and phase 
of gx. 


The COS and SIN Commands 


The TIMESLAB commands x=COS(n,a,p) and y=SIN(n,a,p) generate 
arrays x and y where 


2n(j—1 
RG) x acedae SS Unsonhes 
P 
. 2a(j—1 ; 
Flere 5 A | ee 
P 


and a and p are real scalars with p # 0. 


A data set x(1),...,2(n) is an example of a finite sequence of numbers. 
The basic fact of the harmonic analysis of such a sequence is that we can 
calculate from z(1),...,2(n) a set of orthogonal sinusoids of frequencies 1/n, 
2/n, ..., [n/2]/n, where [c] denotes the greatest integer less than or equal to 
c, together with a constant, so that when the constant and the sinusoids are 
added together, the resulting function coincides with the data at the indices 
1,2,...,n. Further, the sample variance of the z’s is proportional to the sum 
of the squared amplitudes of the sinusoids. 

We will formalize these results in Theorem 1.4.2, but first we have the 
following theorem, which describes the orthogonality of sinusoids. 


Theorem 1.4.1 | ORTHOGONALITY OF SINUSOIDS 


Let w, = (k —1)/n, fork =1,...,[n/2]+1. Then 


n 0, Ww; a Wk 
a) S| cos Qn(t—1)wjcos2m(t—1lhuy,= qn, wi=u, =0,.5 
c=h Nj 2a tei; =O, 0,25 
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{oa Wi = Wr 0,70 


b in 2mr(t — 1)w; sin 2z(t —1 = 
) So sin m( Jw; sin 2n( wk 0, otherwise 


t=1 


c) ) 5 cos 2n(t — 1)w; sin 2x(t — 1)u, = 0, all w;, wr 


t=1 


d) S| cos 2n(t —1)w; = 


t=1 


1a w; =0,.5 


0, otherwise 
e) yo sin 2n(t — 1)w; = 0, all w; 
t= 


Proof: We include the proof because it illustrates techniques used in many 
theorems in harmonic analysis. Letting w, = 0 in part (a) gives part (d), and 
letting w; = 0 in part (c) gives part (e), since cos(0) = 1. We illustrate the 
other parts by proving part (a). Recall the identities 


10 


e’’ = cos@+isin6, See a ee ee) 


yh , 


and denote the real and imaginary parts of the complex number z = a + bi by 
a = Re(z) and b = Im(z). Then 


k= ve cos 27(t — 1)w; cos 2m(t — 1)w, 
t=. 


= ; pea — 1)(w; + wr) + 55 cos 2n(t —1)(; on) 


t=1 t=1 
ea 1 = 2ni(t—1)(wj+we) . 2ni(t—1)(wj—w 
= bre So et-nertn 4 yo at-nteran 
t=1 t=1 
1 n-1 n-1 
ra phe peo ' 


where ry = e?(“i+“k) and rp = e?7(¥i—“*). For a general geometric series 
with base r, we have 


1—r" 
n-1 ‘ Ker 
f=0 n, ied 


Note that r; and rg are both of the form e?"*? = cos270 + isin270, which 
equals one only if @ is an integer. But since w; € [0,.5], the only way w; + uw, 
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can be an integer is if w; = w, = 0 or .5, while w; — w, can be an integer only 
if w; = w,. Thus we have 


ae n, w; =w, =0,.5 
1 
S=don= 1-—rf 
1=0 Serre wes wR OEME Wy = wei 0,5 
1-r, 
sia n, wp Oe 
l 
52 = a Ve 
2 ws Hw, 
1=0 ; P 
1—r2’ 2 


But r? = e2™i(witwe)n — e2niG+k-2) — 1 and r? is also one, so that the second 
terms for S; and S> are both zero. Thus, 


1 
Sik gRe(Si + So) 


nd; — We —0,.5 


Ul 


x wai wy, Oyo 
0, wp x wks 


which was to be proved. 


Before presenting the sinusoidal decomposition of a time series data set, 
we define the discrete Fourier transform of a set of numbers. 


Definition. The discrete Fourier transform (DFT) of the (possibly complex) 
numbers z(1),...,2(n) is the set of complex numbers z(1),...,z(n) given by 


z(k) = eyaa(eje" ee” te 


t=] 
n n 
ms ye z(t) cos 2r(t — 1)wR + vy z(t) sin 27(t — 1)u,, 
t=1 t=1 


where w, = (k—1)/n, k =1,...,n. 


If we insert a minus sign into the exponent in the first expression, we have 
what is called the inverse discrete Fourier transform (IDFT), since if one were 
to find the IDFT of the DFT of z, the result would be nz. 
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The basic facts about the harmonic analysis of z(1),...,2(n) are given in 
the next theorem, the results of which can be shown using Theorem 1.4.1 and 
further trigonometric identities. 


Theorem 1.4.2 | SINUSOIDAL DECOMPOSITION OF DATA 


Let x(1),...,2(n) be a finite sequence of numbers and let z(1),...,2z(n) 
be the DFT of z. Fork =1,...,n, let w, = (k — 1)/n and define 


Ge Foe (MLL bye = ea 


n n 
Let 6? = 157", (x(t) — @)”, and fort =1,...,n define 


gk (t) = ap cos 2m(t — 1)w, + by sin 2x (t — 1)w, 
= C; cos[2m(t — 1)w, — arctan(b, /a,)]. 


Then 

a)'ay = 2, b, = 0, Gea gilt)=c 

b) x(t) — = Yop ao ge(t) 

c) 6 = Vpn OF 

d) Fork =2,...,[n/2]+1, we have 
ak = Gn—k+2; bk = —bn—k+2 
cos 2m(t — 1)w, = cos 2mWp_K42 
sin 2m(t — 1)we = —sin 27Wpn_K+2 


and thus 
gx (t) = 9n—k+2(t), CR = Coes 


which gives 


2 yin/al+ (t), todd 
z(t) ei re 
2 k=? 9x (t) + 9(n/2)+1(t), n even 
ie Nee CR, n odd 
oz 
eae! C2 + Con /2)419 n even. 
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e) For k = 1,...,[n/2]+ 1, the vectors gy = (gx(1),---,9x(n))? are or- 
thogonal, that is, Sj Bk eed edeFanks 


Implications: This theorem gives a decomposition of a time series data set 
into a set of orthogonal (part (e)) sinusoids of frequencies 1/n, 2/n, ..., [n/2]/n 
(part (d)), and gives a decomposition of the variability in the x’s (as measured 
by &?) in terms of the sum of squared amplitudes of the sinusoids (part (c)). In 
terms of understanding why the z’s vary then, these squared amplitudes play 
an important role. Note that C? = |z(k)|?/n?. 


Definition. For a time series data set z(1),...,2(n), let w, = (k — 1)/n, 
k=1,...,[n/2]+1, and define 


C= Z [Soa (tye2rieDee k=1,...,[n/2]+1. 
t=1 


ae 
A plot of nC? versus w,, is called the periodogram of xz. The function 


in “|Dha ae], w €[0, 5] 
()) — 
f(l—w), w €[5,1] 


is called the sample spectral density function of z. 


Note that the periodogram is the sample spectral density evaluated at the 
so-called natural frequencies wy, we, ...,; W[n/2]41- 


Interpreting the Periodogram 


In Figure 1.5 we give a plot of three data sets of length 200 (and the log 
of their periodograms) that were constructed by 


z(t) = acos 2n(tasl} + Bcos 


2n(t — 1) 2n(t — 1) 
100 —— + dcos 


10 4 ‘ 

that is, as the sum of pure cosines of frequencies 1/100, 1/10, and 1/4. The 
three series were obtained by varying a, @, and 6 (Series 1 has (10,3,1), Series 
2 has (3,3,3), and Series 3 has (1,3,10)). 

Recall that a sinusoid of frequency w is the sum of a sine and a cosine of 
that frequency, but for simplicity we are considering sinusoids that have no sine 
part. A sinusoid of long period (low frequency) is very smooth in appearance 
relative to one of short period (high frequency). Thus when a is large relative 
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Figure 1.5. Sums of Pure Cosines and Their log Periodograms. 


to @ and y (such as Series 1), we would expect z to be relatively smooth in 
appearance; that is, the long-term rise and fall of the data should be large 
relative to short term oscillations. On the other hand, Series 3 appears quite 
wiggly since y is large relative to a and @. In Series 2 we have used equal 
values of the three coefficients and the resulting data are between Series 1 and 
3 in terms of wiggliness. 

Note how the periodograms pick out the amplitudes of the component 
sinusoids. In general, of course, time series data sets are not made up of sums 
of only a few pure sinusoids. However, with the above example as motivation, 
we can make the qualitative statements given in Table 1.2. 


In Figure 1.6 we give plots of the logarithm of a standardized form of 
the periodograms of the nine univariate series that we have been discussing 
throughout this chapter. Note how these graphs confirm the discussion above. 
In particular, note the harmonics in the periodogram of the airline data, the 
random pattern in those for Series III and IV, the excess of low frequency in 
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Table 1.2. Interpreting the Periodogram 


Appearance of Data Nature of Periodogram 


Smooth Excess of low frequency; that is, amplitudes of 
sinusoids of low frequency (long period) are 
large relative to other frequencies 


Wiggly Excess of high frequency 
Random (no pattern) No frequencies dominate 


Basically sinusoidal A peak at frequency 1/p 
of period p time units 


Periodic of period p A peak at fundamental frequency 1/p and 
but not sinusoidal peaks at some multiples of 1/p (harmonics) 
(see Problem C1.6 for an example of this) 


Series VII, the periodicity in the birth data, and how the periodogram is able 
to tell that Series IX is in fact the sum of four pure cosines. 


Displaying the Periodogram 


It often happens that a few values of a periodogram are very large relative 
to the rest and thus dwarf them in the plot. Plotting the (natural) log of 
the values rather than the values themselves allows us to see other possible 
values of interest. In Chapter 3 we will see other reasons for plotting the log 
(see Theorem 3.1.4). We would also like to plot the log periodogram on some 
standard scale so that several such plots can be compared in a meaningful 
way. A useful standardization of the periodogram is suggested by the following 
theorem, which follows immediately from part (c) of Theorem 1.4.2. 


Theorem 1.4.3 | STANDARDIZING THE PERIODOGRAM 


Let nC? for k = 1,...,n be the periodogram ordinates of a time series 


z(1),...,2(n), and let 6? = * 3, (a(t) —z)?. Then 


that is, the average value of nCZ/6? is one. 
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Figure 1.6. The Log of the Standardized Periodograms of Series I-IX. 
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Because of this theorem we will routinely display 


nC? k-1 
log (“) versus ——, k= 1... 2, (n/2]+ 1, 


with the graph truncated at —6 and 6 on the vertical axis. We will rarely 
encounter a value of nC?/é? that is greater than e® or less than e~°. 


The PLOTSP and PLOTCSP Commands 


If one has an array f of length (Q/2]+1 and a real scalar c, then the 
command 


PLOTSP (f ,Q,c) 


will produce a plot of log(£(j)/c) versus (j — 1)/Q for j = 1,...,[Q/2]+ 1, so 
that the vertical scale runs from —6 to 6. The command 


PLOTCSP(f ,Q) 


will produce a plot of f(j) versus the same set of frequencies with no truncation. 
Thus one can use PLOTSP (with c being G7) to get the standard periodogram 
plot that we have been discussing, or use PLOTCSP to get a plot of the actual 
periodogram. These two commands will be used extensively in the sequel, as 
we will be introducing other functions that will be similar to the periodogram. 

It is important to note that most of the TIMESLAB commands that pro- 
duce arrays to be plotted via PLOTSP or PLOTCSP will have Q (the number of 
frequencies between 0 and 1) as an argument but will produce only [Q/2]+1 val- 
ues because the quantities that are calculated are symmetric about frequency 


1 
The Sample Spectral Distribution Function 


We have seen that data that have no obvious trends, cycles, or serial corre- 
lation should have a periodogram that oscillates roughly about a constant. On 
the other hand, smooth (wiggly) data have an excess of low (high) frequency, 
while cyclic data will have peaks in their periodogram. One useful way to 
display these ideas graphically is via the sample spectral distribution function. 


Definition. Let f(w;),.. ., f (wg) be the sample spectral density function of 
data z(1),...,a(n) at the frequencies w; = (j —1)/Q, j = 1,.-.,Q... Let 
q = [Q/2]+1 and 


k f . 
Pu) = Fs) k=1,...,9. 


et f(w;) 


22 DESCRIBING AND TRANSFORMING SERIES CH. 1 


Then F(w),..., F (wg) is called the sample spectral distribution function of z. 


_ IfQ=n, then the f’s are in fact the periodogram ordinates, and thus the 
F’s could be called the cumulative periodogram of z. Note that F(w,) = 1, 


while if Z = 0, we have f(w,) = 0 and so F(w,) = 0. Thus 
0 < F(w,) <1,. « kK=1,<:5,4, 


and the spectral distribution function of random data will fluctuate about the 
straight line that connects the points (0,0) and (.5,1), that is, the line y = 2z. 


The CUMSP Command 
The command 
F=CUMSP (f ,Q) 


will compute the sample spectral distribution function F corresponding to the 
sample spectral density function f. Again, note that Q is the number of fre- 
quencies between 0 and 1 at which f has been calculated, while the arrays F 
and f are of length qg = [Q/2]+ 1. Once F has been found from CUMSP, then 
PLOTCSP can be used to plot it. 

In Figure 1.7 we display the cumulative spectral density of our example 
data sets. Note that these graphs are much less variable than their correspond- 
ing log periodograms. Also, an excess of low (high) frequency means that the 
cumulative periodogram starts out above (below) the line y = 2z before catch- 
ing up to it before w = .5, while peaks in the periodogram translate into jumps 
in the cumulative periodogram (such as at frequency 1/12 in the airline data), 
and the cumulative periodograms of the white noise series (Series III and IV) 
vary little from the line y = 22. 


1.4.4. The Relationship of the Correlogram and Periodogram 


Recall that the sample autocorrelation coefficient of lag v for data z is 
given by 


roy! (x(t) — 2) (2(t + v) — 2) 


p(v) = =", (a(t) - 2) 


Ju] <n. 


Then f(v) can be written as 
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Figure 1.7. Cumulative Periodograms for Series I-IX. 
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if we define the sample autocovariance function R by 


n—|v| 


R(v) = — S> (2(t)—#)(z(t+v)-2), ol <n. 


v= 


We show in this section various relationships among /, R, and the sample 
spectral density f. We also describe how an algorithm called the fast Fourier 
transform (FFT) can be used to calculate the correlogram and periodogram 
much more rapidly than a straightforward implementation of their defining 
formulas. These results are summarized in the next theorem. 


Theorem 1.4.4 | RELATIONS Amonc #, R, f 


Let x(1),...,2(n) be a time series data set, and for t = 1,...,n, let 
z(t) = x(t) — z. For a positive integer M, let 


Z\h hy tee 
uo=| 
0, t=n+1,...,.n+M; 


that is, the series y is just the series z “padded” with M zeros. Let pz, Re 
and f, be the autocorrelation, autocovariance, and sample spectral density of 
z, respectively, and let f, be the sample spectral density of y. Then 


a) f.(w) = > Ra(ve"™™, — w € [0,1] 


Jul<n 


a 1 a . 
b) i, (v i; fila\ep =, lul<n 


Q 
c) R,(v) = 9 hl (wy )e2™Ire , (ean a 
where wz = (kK —1)/Q, k = 1,...,Q. 


Implications: Before proving this theorem we note that parts (a) and (b) 
show that f, and R, are related in a very special way, as what are called 
Fourier transform pairs. Also, part (c) shows that the sample autocovariances 
can be calculated as the DFT of the sample spectral density (which is itself the 
square modulus of a DFT) of the padded series y. This is important because 
in some circumstances doing two DFTs (one to get fy and one to get R, ) can 
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be much faster than calculating R, directly from its defining formula. We will 
discuss this further when we introduce the fast Fourier transform. 


Proof of Theorem 1.4.4: To prove part (a) we note that 


a 2 


a zijene 


t=1 


a (3 2(t)e Qni(t— 1) (U (s)en2mi(e—2), 


nf, (w) = 


since the complex conjugate of this first term in parentheses is in fact the 
second term. Thus, 


nfz( (w) = ys S>a(t)a(s Wen ores 
t=1 s=1 
Instead of doing this double sum by letting ¢t and s vary freely, we will add 
the terms for a fixed value of s —¢ and then add the results for the various 
possible values of v = s —t. This is the discrete analog of change of variables 
in integration. Note that s —t ranges from 1—n to n—1. There are n — |v| 
terms in the double sum that have s — t = v, namely, the terms 


2(1)z(1 + ler. Seats z(n = \ul)z(n)e27*, 
Thus, 
£ ae n—|»| . 
fz(w) = = Pass Due z(t)z(t+ amie 
v=—(n-1) t=1 
—|vl 
1 n 
=s > (— > Zz (t)z( (t+ vl) )e —2nivw 
v=—(n-1) 
n-1 * 
— by R,(v)e7?7, 
v=—(n-1) 
since zZ = 0. 


To prove part (b) we can now write 


n-1 


Hs futujerio =f > fu wecian are 


k=-(n-1) 


n-1 e 1 : 
aa Ne R, (k) | e2ti(u—k)w da. 
0 


k=-—(n-1) 
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But this integral is zero unless k = v in which case it has value one and this 
fact gives part (b). For part (c), note that 


: R,(v), lvl <n 
,(v) = 
0, |u| 2 n, 
and thus by part (a) 
1 Q n—1 


Pa Qrivw, 1 2 DP (4\p—20tjw, \ ,2TivW, 
@ flee =5( Yo Relient") 


k=1 j=-(n-1) 


n—-1 P Q ) ; pees) 
=) Wail Dons ie 
j=—(n—-1) k=1 


and this last sum is again, as in the proof of Theorem 1.4.1, a geometric series 
with r = e27#(¥-3)/2_ Again, this sum is zero unless (v — j)/Q is an integer, in 
which case it has the value Q. But the only way that v — j can be a multiple 
of Q is if v = j, since the most that v and j can differ is Q — 1 because 
—(Q-—n)<v<Q-nand —(n—1) <j <n—1. Thus the only term in the 
sum over j that is nonzero is when j = v, which is in fact R,(v), which was to 
be shown. 


The Fast Fourier Transform (FFT) 


To calculate the discrete Fourier transform 


w(k) = Nl eg era hires Uns Thy 


t=1 


of data z(1),...,2(n) appears to require n? multiplications and additions (n 
for each of the n values of k) and many evaluations of complex exponentials. In 
the mid-1960s various researchers made use of a variety of trigonometric iden- 
tities to obtain algorithms called fast Fourier transform (FFT) algorithms (see 
Brigham (1974)) that essentially require only n(p; + --- + px) multiplications 
and additions, where pj,..., pz are the prime factors of n, and a greatly reduced 
number of evaluations of complex exponentials. For example, ifn = 1024 = 21°, 
then the number of operations is 1024(2+---+2) = 10-2-1024 = 2log, 1024, 
which is approximately 50,000 as opposed to 1024? ~ 1,000,000, a savings of 
a factor of about 50. In Section A.3 we describe some of the details of how the 
FFT works. 

Note that if n is not very composite, that is, it has some large prime 
factors, the FFT saves very little over a straightforward DFT. The subroutine 
that is used in TIMESLAB for calculating the FFT requires that n meets two 
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Figure 1.8. Sample Spectral Density of Series IX at 256 Frequencies. 


requirements. First, the largest prime factor of n must be less than or equal to 
23. Second, the product of what are called the “square-free” prime factors of 
n must be less than or equal to 210. For example, the product of the square- 
free prime factors of n = 1260 = 2? -3?-5-7 is 5-7 = 35, which meets the 
requirements. If the user tries to use a command that asks TIMESLAB to 
find the DFT of length n and n does not meet the above requirements, then 
a message is displayed on the screen and the command aborts. Sometimes a 
user can write a macro that performs the same task as a command but calls 
the FFT command explicitly. This would allow the task to be performed even 
though it may take a very long time. In almost all cases padding with zeros to 
increase n is sufficient. Note that Series I (n = 365), II (n = 370), and III and 
IV (n = 106) each had to be padded with zeros before finding their Fourier 
transform, and thus Figure 1.6 actually contains graphs of the sample spectral 
density for those series. In most cases, the two graphs (periodogram and sample 
spectral density) will look very similar, that is, the features of the periodogram 
(excess of low or high frequencies or existence of peaks) will also be present 
in the sample spectral density. To illustrate this, consider Figure 1.8, which is 
the sample spectral density of Series IX, the sum of four cosine curves. The 
four curves were designed to have periods that all divide n = 144, and thus the 
amplitudes of all frequency components in the DFT for 144 frequencies are zero 
except for frequencies 1/36, 1/12, 1/6, and 1/3. However, if we were to find the 
DFT for Q = 256, as in Figure 1.8, none of the amplitudes would be zero, and 
the amplitudes for the important frequencies would not be calculated. Note 
that there are still four sharp peaks in the sample spectral density. 
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The FFT Command 

The TIMESLAB command 

FFT(zr,zi,n,m,sign,wr,wi) 


calculates the DFT (if sign=1) or inverse DFT (if sign= —1) of the n complex 
numbers having real and imaginary parts zr and zi and stores the real and 
imaginary parts of the first m values of the result in the arrays wr and wi. We 
have up to now considered only Fourier transforms where the input is real. We 
will have occasion later to need the transform of complex arrays as well. The 
command 


FFT(z,n,m,sign,wr,wi) 


is the same as the form above except that zi is taken to be zero. 

If n is not composite enough, the FFT command will display a message 
stating that the FFT algorithm is not available, and will use the defining 
formula to calculate the transform. A message is displayed warning that the 
command may take a long time to execute. 


The FFT and Convolutions 


Let z(1),...,2(n) be a data set such that z = 0. The FFT is useful in 
calculating the autocovariance function R,, which is defined to be the convo- 
lution 

1 n—|»| 
Re(v) =— D> z(t)e(t+ lol), bel <n. 


i) 


If we use this formula to find R,(0),...,R,(M), we need 


n+(n—1)+---+(n—M) = (M+ 1n—- MD 
multiplications and additions. If, on the other hand, we pad z with r > M 
zeros such that Q = n+r is the next power of 2 greater than or equal to n+ M, 
then we can use part (c) of Theorem 1.4.4 to get the desired quantities by doing 
two FFTs of length Q, that is, in 2Q log,(Q) multiplications and additions plus 
some evaluations of complex exponentials. As an example of this, suppose n = 
1000 and M = 100. Then if Q = 2048, we would require 2(2048)(11)=45,056 
operations to do the two FFTs versus 101,000—5,050=95,950 operations using 
the convolution formula. In Example 1.9 we give a macro that allows one to 
experiment with which method is faster for various values of n and M. 
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The CORR Command 


The CORR command is flexible in that it can calculate just correlations, just 
the sample spectral density, or both at the same time. If both are asked for, 
CORR uses the two FFT method for finding correlations and does the padding 
with zeros automatically. The input data array is not affected in any way by 
the command. Thus forQ >n+M andM>0O, 


rho=CORR(x,n,M,Q,iopt,RO,f) 


will calculate ro=t ye te? (t) and 
n 


nm—vU 
z(t)z(t+ v) 
rho(v) = ‘St RO d v= i lal. 
and 
1\< P 
£G)= = Se z(te?ME-DG=I8 j=1,...,[Q/2] +1, 
t=1 


where z(t) = x(t) if iopt=2 or z(t) = x(t) — z if iopt=1. 
To get the first M correlations and the periodogram (as opposed to the 
sample spectral density function above), CORR must be called twice: 
rho=CORR(x,n,0,n,iopt ,RO,per) 
to get the periodogram, and 
rho=CORR(x,n,M,0,iopt ,RO,per) 
to get the correlations. To get RO and nothing else, one can use 


rho=CORR(x,n,0,0,iopt,RO,per). 


If one or more of the output arguments is not calculated, then no such variable 
is formed by TIMESLAB but some name must be placed in the argument list 
as a “place-holder.” 


1.5. Transformations 


Some of the time series analysis techniques that we will introduce in later 
chapters assume (1) that the data being analyzed have no deterministic trends 
or cycles, and (2) that the variability in the data is constant over time. The 
traditional method of analyzing data that fail to meet these requirements is to 
do the analysis in three steps: (1) try various transformations until the result 
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appears to meet the requirements, (2) analyze the result of step 1, and (3) do 
the inverse operation of what was done in step 1. This strategy is very similar 
to that used in regression analysis. In this section we will describe some of the 
transformations that are often used in time series analysis. 


1.5.1. Stabilizing Variance 


Suppose that the variability in a data set xz appears to be increasing as 
time increases (see the airline data for example). If the mean level of the data is 
also increasing with time, then the variability in y(t) = log z(t) should appear 
fairly constant over time. This is a fairly common occurrence in real data. In 
general, one might try various power transformations to obtain a series having 
constant variability; that is, 


u(t) = (z(t))”. 


Note that in TIMESLAB it is easy to raise each element of an array to a power 
r by using the Type 2 command y=x*~r, and that the command y=LOGE(x,n) 
will find the natural log of an array. Also, for a positive number c, 


log, c 


lo810° = ioe 10’ 


and so we can find the base 10 logarithm of an array x by 


y=LOGE(x,n) 
ee=LOGE(10) 
y=y/ee 


1.5.2. Removing Trends 


Another common phenomenon in time series data is that the values appear 
to be growing in some polynomial fashion with time, particularly linearly; that 
is, the data appear to follow 


z(t) =a+bt+ e(t), 


where ¢(t) is white noise. Such polynomial trends are often removed by using 
differencing. 
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Definition. The dth difference of a time series data set x having n elements 
is a data set y having n — d elements obtained by 


y(t) = a(t + d) — 2(t), t=1...,n— A. 


To illustrate the use of differencing, suppose that z(t) = a+bt+e(t). Then 
y(t) = a(t + 1) —2(t) =a + bt + c(t +1) — [a+ b(t — 1) + €(0)] 
= b+e(t+1)-€(t), 


which no longer has a linear trend. It is easy to see that if z contains a general 
dth-degree polynomial trend, applying first differencing d times will remove it, 
while if x has a cycle of length s time units, then taking sth differences will 
remove the cycle. 

Perhaps the most well known example of differencing is the airline data 
that we have been analyzing. The variability in the data is increasing with time 
and thus a log transform is usually applied. The result of this transform has 
an obvious 12-month cycle and also a linear trend. Thus the traditional advice 
on a series such as this is to take the first difference of the 12th difference of 
the log of the original data. 


The DIFF Command 
The command 
y=DIFF(n,d,x) 


will form an output array y of length n—d as the dth difference of x. Thus the 
commands 


y=LOGE(x,n) 
y=DIFF(n,1,y) 
nmi=n-1 
y=DIFF(nm1,12,y) 


will form the first and 12th difference of the log of an array x. Note that the 
resulting array will be of length n — 13. 

Applying differencing is very simple and often leads to good results, but 
some care must be taken. For example, in our linear trend plus white noise 
example given above, the result of taking first differences was 


y(t) =b+e(t+ 1) -—€(t), 


which is a special kind of time series called a moving average time series (see 
Section 2.5.3) of order one with coefficient —1. Some standard analysis tech- 
niques cannot be applied to such a time series. 
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4.09 session Residuals 
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Figure 1.9. Residuals from Regression for Series V. 


Regression Analysis 


A traditional method of removing trends and cycles from time series data 
is to do ordinary regression of the data on the deterministic functions of time. 
For example, Series V of the univariate series that we have been analyzing was 
formed by 


2n(t—1 
z(t) = 104 It + 3cos U1) 


where €(t) is a series having serial correlation (actually €(t) is a realization from 
an AR(1) process with coefficient —.75; see Section 2.5.4). If we did ordinary 
least squares regression of the form 


2n(t — 1) 


y(t) = Bo + Bit + Bo cos 0 


+ e(t), 


we would obtain the residuals given in Figure 1.9, which can then be further 
analyzed by time series analysis. See Example 1.10 for the macro that generated 
this figure, and Section A.5 for a complete description of the types of regression 
analyses that are available in TIMESLAB. 


1.5.3. Accounting for Seasonal Variability 


Often it is of interest in seasonal time series to analyze how a data set 
differs from regular seasonal variation. For example, suppose that z consists 
of m years of monthly data, that is, n = 12m. We can define the monthly 
means and variances to be the means and variances of each of the 12 data sets 
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consisting of like months; the Januaries, Februaries, and so on. Thus, 


Ip = 4 ye (b+ 12(¢ 1) 


™ 


= — 5 [ek +12¢-1) - a] , 


t=1 


k = 1,...,12. Note that other seasonal (quarterly or hourly, for example) 
means and variances can be defined similarly. See Problem T1.7 for more 
information about seasonal means. 


The SUBMNS and DIVSDS Commands 


The SUBMNS command will calculate and/or subtract seasonal means from 
a data set. It can also be used to add back some seasonal means that were 
previously subtracted. This command is also used to just calculate and/or 
subtract the “overall” mean of a data set, that is, the mean of the entire data 
set. Thus, 


y=SUBMNS(x,n,d,xbar) 


will calculate and subtract seasonal means xbar of seasonality d for an array 
x and return the result in y. The form 


y=SUBMNS(x,n,d,xbar,iopt) 


with iopt=1 is the same as that above, while if iopt=0, then xbar is formed 
but y is not, and if iopt=2, then an inputted array xbar is added to x to get 
y. 

The command DIVSDS operates the same as SUBMNS except that it refers to 
calculating and/or dividing or multiplying seasonal standard deviations, that 
is, to square roots of the seasonal variances defined above. Again, if d=1 in 
any of the forms above, then the command operates on the overall mean or 
standard deviation. 


1.5.4. General Smoothing Operations 


Another approach to removing deterministic-looking parts of a time series 
is to use general methods of smoothing data and then analyze the deviations 
from the smooth version. For example, one way to smooth wiggly data is to 
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Data (dotted) and Smoothed Data (solid) 
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Figure 1.10. An Example of the Moving Average Smoother. 


use a moving average smoother, for example, the one of length 3: 


y(1) = 3 
y(2) = z(2) + 2) + x(4) 
hee a(n — 2)+2(n—1) +2(n) 


3 


Since consecutive y’s have two of the x’s in common in their average, we would 
expect that the y’s won’t vary as much as the original z’s; that is, y will be 
much smoother in appearance. In Figure 1.10 we have generated 100 points of 
a cosine of amplitude one and period 40, added N(0,1) white noise to it, and 
then used a moving average smoother of length 11. Note that the last smoothed 
value corresponds to the 95th data point, and so we have superimposed the 
6th through 95th data points and the smoothed data. Note that the smoothed 
data clearly exhibit the cosine curve, which is not obvious in the original data. 
See Example 1.11 for the macro that formed this figure. 


The FILT Command 


The moving average smoother is a special case of what is called a linear 


filter: 


y(t) = Boz(t + m) + Biz(t +m—1)+---+ Bnz(t), . 1a. ht 
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The command 
y=FILT(x, beta, beta0,n,m) 


will perform such a filtering operation where beta is an array containing (;, 
..., Bm, and beta0={y. We will discuss linear filters in detail in Section 2.3. 


1.6. Some Simple Forecasting Methods 


An important objective of time series analysis in many scientific areas is to 
forecast (predict) future values of a time series z(1),..., 2(n). In this section we 
begin our discussion of forecasting methods by looking at some simple methods. 
In Section 2.4 we study the theory of prediction, and in Chapter 3 we discuss 
some of the more elaborate methods that are used in practice. 

Each of the methods in this section can be viewed as having two parts: 
(1) a model-fitting part, and (2) a forecasting part. Note that many statistical 
procedures are of this form. 


1.6.1. The Inverse Differencing Method 


A simple but effective method for extrapolating trends and cycles in data 
that can be handled by differencing is to extend the data set with M new values 
in such a way that if the extended series of length n+ M were then differenced, 
the last M values (the ones corresponding to the new values) would all equal the 
mean of the differences of the original data. For example, if a data set appears 
to have a linear trend, we might detrend it by applying first differences. We 
could then find a new data point z(n + 1) so that (n+ 1) — x(n) = Z, where 
zZ is the mean of the differences of z(1),...,2(n). Thus x(n +1) = z(n) +2, 
and if we continue this process, we obtain 


a(n+h) =2(n+h—1)+4+z2, hel} 


If we have data that appear to contain both a linear trend and a seasonal 
cycle, we might apply both first and dth differences, where d is the length of 
the seasonal cycle. For monthly data having an annual cycle, we would solve 


z(n + 1) — a(n) —2(n— 11) + 2(n — 12) =z, 


where Z is the mean of the first and 12th differences of z(1),...,2(n). This 
results in the predictor 


z(n+h) =2(n+h—1)+2(n+h— 12) —a2(n+h-— 13) +2, h>sl, 
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The EXTEND Command 


Given an array X of length n containing a data set, an integer di, and 
optionally another integer d2, the command 


y=EXTEND(x,n,next,di[,d2]) 


will carry out the process of extending an array by the inverse operation of 
differencing as described above. The first n elements of the array x will be 
extended by next points and the result will be placed in the array y which 
will be of length n+next. One or two differences can be specified depending on 
whether or not the argument d2 is included. 

To illustrate the use of the EXTEND command, consider Figure 1.11, which 
contains the result of the commands 


READ(air,x,n) 

y=LOGE(x,n) 

yext=EXTEND(y,n,24,1,12) 

np24=n+24 

yext=EXP (yext ,np24) 

LABEL(yext)=’Airline Data and Extended Values’ 
LABEL(x)=? ? 

PLOT2(x, yext,n,np24,2,1,0,170,0,800) 


The observed data values are represented on the plot by both a solid curve 
and an x at each point, while the extended values are represented only by the 
solid curve. The extended values are the last two cycles on the graph and are 
consistent with what we would expect from a predictor. It is not surprising that 
we could use this “naive” method on the airline data as it is a long-memory 
series and we would expect that almost any method would work well, including 
just drawing in the next two cycles by hand. For series that do not follow such 
a deterministic pattern we will have to use more sophisticated methods. 


1.6.2. The Regression Method 


If the data appear to follow a deterministic function of time that can 
be expressed as a simple or multiple linear regression model, we can predict 
future values of the series by substituting future values of t into the least 
squares regression function (see Section A.5 for a discussion of multiple linear 
regression). For example, if 


z(t) =a+bt+ icon EL + e(t), 
P 


for t = 1,...,n, where the period p is known, we can estimate a, b, and c by 


their least squares estimates a, 6, and ¢, and forecast a value A steps into the 
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firline Data and Extended Values 
800.08 
720.0 
648.88 
568.08 
480.00 
403,08 
328.8 
248.08 
168.0 
80.0 


0.0 
0.0 34.0 68.00 102.0 136.0 170.0 


Figure 1.11. Airline Data and Extended Values. 


future by 
Z(n+h) Be bf ean) OE eee tee 
Pp 


1.6.3. Simple Moving Average 


This method models an observation as a simple average of the previous m 
observations where m is to be chosen. For a given moving average length k we 
can calculate 


as a measure of how well the simple moving average model of length k fits 
the observed data. We chose m as the value of k minimizing S(k). Then we 
forecast future values recursively. For example, if m = 3 we calculate 


E(n+1)= 5 [2(n) +z2(n—1)+2(n— 2)} 
er oye 5 [a(n sa aris Cees (ae 
Bw HB) feta 419) We Genre (n)))! 


3 


and so on. 


38 DESCRIBING AND TRANSFORMING SERIES CH. 1 
1.6.4. Simple Exponential Smoothing 


Instead of modeling an observation as a simple average of the previous m 
observations, exponential smoothing methods model x(t) as a weighted average 
of all of the previous values. The type of weights that are used depends on 
the appearance of the data and leads to methods having a variety of names. 
We will discuss only the simple exponential smoothing technique which is most 
suitable for data that appear to have no linear or seasonal trends. 

The simple exponential smoothing method consists of modeling x(t) as a 
weighted average: 


t 
(t+ 1)=) Bj2(t+1-3), © #21. 


If we let 8; = a(1 — a), where 0 < a < 1; that is, we let the weights decay 
exponentially to zero, hen ye =i); =1, ard thus for large t, the weights will 
approximately sum to one. We can choose a as the value of a minimizing 


n t-1 
S(a) =D | ()— Dall — ae 3) ] 
t=2 j= 


and then forecast the value at time n+ 1 by 


n 


&(n+1)= Yo a(l —a)—tz(n+1—J). 


g=1 
The calculations involved in this process can be greatly reduced by noting that 
#(t+1) = az(t) + (1—a)az(t), 


where £(1) is defined to be 2(1). We can also write this as 2(t+1)—(1—a)z(t) = 
az(t), which is called a difference equation of order one. Difference equations 
are very important in the study of time series analysis. 


1.6.5. Difference Equations and the Behavior of Forecasts 


Definition. Let z(-) and w(-) be sequences of real numbers. Then 
z(t) + ayz(t —1)+---+ apz(t — p) = w(t) 
is called a difference equation of order p, coefficients aj,...,@p), and forcing 


term w(-). 


To calculate the values of z for all values of t it is sufficient to know 
all of the values of w and any p consecutive values of z. These p values are 
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called starting values or initial conditions. If we know the starting values 


z(1),...,2(p) and the values w(p+1),...,w(n) for a difference equation, then 
we can find z(p+1),...,z(n) recursively by 


P 
2(p+j)=w(p+j)— > anz(ptj—-k), j=l,...,n—p. 
k=1 


Note that w(p+1) and z(1),...,z(p) are used to find z(p+1), which is in turn 
used in finding z(p + 2), and so on. 


The ARDT Command 
In the command 


z=ARDT(alpha,p,n,w) 


the array alpha contains a,...,@p, while the array w has z(1),..., z(p) as its 
first p elements, and w(p+1),..., w(n) as its last n — p elements. The output 
array z contains z(1),...,z(n). Thus in 


z(t) — (1 — a)z(t — 1) = az(t — 1) 


we have p = 1, z(t) 


= £(t), z(1) = 0, ay = —(1—a) = a—1, and w(t) = az(t—1). 
Thus we can find £(1),. 


@(n) and S(a) = yp» (#(t) — a)’ b 


es 


oa=a-1 

alpha=<oa> 

w=a*x 

w=<x[1] ,w> 
xhat=ARDT(alpha,1,n,w) 
e=x-xhat 

Sa=D0T(e,e,n) 


In Example 1.12 we give a macro which will choose a and superimpose the 


plots of the data and the resulting fitted values. Note that we can also write 
the simple exponential smoothing recursion as 


a(t +1) = a(t) + a(2(t) - 8) ; 
that is, the predicted value at time t + 1 is that for time ¢ plus an adjustment 


for the error in using £(t) to model z(t). To forecast x(n + h) for h > 1, some 
authors suggest using 


&(n+h) = a(n) + (l—a)é(n+h—-1). 
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Many of the forecasting methods used in time series analysis produce fore- 
casts that are future values of a difference equation. In Theorem 1.6.1 we 
present results that can be used to describe analytically the behavior of these 
forecasts. The results of this theorem will be useful in other contexts as well. 


Definition. Let 


S~ aj2(t — j) = w(t) 


j=0 


be a pth-order difference equation. If w(t) = 0 for all t, then the difference 
equation is said to be homogeneous. The polynomials 


P P 
q(z)= Sage and A(z)= YE Gah 
g=0 7=0 


are called the characteristic and indicial polynomials, respectively, of the dif- 


ference equation. 


Let z1,...,2p be the zeros of h; that is, h(z;) = 0. Note that the zeros of 
g are the reciprocals of the zeros of h. These zeros play an important role in 
solving difference equations. 


Theorem 1.6.1 |SOLUTION OF DIFFERENCE EQUATIONS 


Suppose that z(t) satisfies a pth-order homogeneous difference equation, 
and let z,,...,Zp be the zeros of the indicial polynomial of the equation. Then 


a) The zeros are either real or occur in complex conjugate pairs; that is, 
if z; = a+ bi is a complex valued zero, then z; = a — bi Is also a zero. 


b) z(t) is equal to the sum of terms of two types. If a real root z occurs 
m times, then a term of the form 


(Ar + Bot +---+ Bom a 


is included. For a distinct real root this term is #,z*. If a conjugate pair 
(z = a+ bi, Z =a — bt) occurs m times, then a term of the form 


[11 cos(t + 61) + yot cos(tO + 52) + -+-+ Ymt™* cos(tO + bn)]ré 


is included, where r and @ are determined by expressing z and Z as r(cos@ + 
isin@); that is, r = Va? +b? and 0 = arctan(b/a). In either case, the 6’s, 
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y’s, and 6’s are determined by solving the system of equations resulting from p 
initial conditions, that is, by setting any p known values of z equal to the sum 
of the terms described above. 


c) If the zeros are al] real and distinct, then as t + 00, the values of z(t) 
either converge to zero, oscillate between —oo and oo, or diverge to oo. 


Note that part (c) is just one possible application of part (b) (see Problem 
T1.11 for another). To illustrate solving a difference equation analytically, 
consider 


2(t) — z(t — 1) + z(t — 2) — z(t — 3) =0, t>4, 


with initial conditions z(1) = 1, z(2) = —1, and z(3) = 1. Then the indicial 
polynomial is 
h(z) = 27 — 2? +2z—-—1= (2? + 1)(z—1), 


which has zeros z; = 7, z2 = —i, and z3 = 1. For z3, a term of the form @1' = 6 
is included, while for z; and z2, we have r= 1 and @ = 7/2, and thus a term 
of the form y cos(t1/2-+ 46) is included. Thus, 


t 
z(t) = B+ ycos(—= +8), 
where (7, 7, and 6 are found by solving the equations 
aT) se 7108(5 + 4) 
2(2) = -1= 6+ ycos(x + 6) 
3 
ZS yee bee B + ycos(-> +6). 
These equations are satisfied by @ = 1, y = —2, and 6 =z. This gives 
z(t) =1- 2 cos(— +7), 


and thus z repeats its values every four points, with z(4) through z(7) given 
by 3,1,—1, and 1. We can verify this numerically by the commands 


alpha=<-1,1,-1> 
f=LINE(100,0,0) 
f=<15=154i4tS 
x=ARDT(alpha,2,100,f) 
LIST(x, 100) 


If we were to plot the real and imaginary parts of a complex number 
z= a+bi on an X-Y plane, then |z| = Va? + b? is greater than one if and 
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only if the plotted point falls outside of a circle of radius one that is centered at 
the origin. This circle is called the unit circle, and we will henceforth refer to 
zeros being inside, outside, or on the unit circle. In Section 2.6 we will describe 
methods for determining whether the zeros of a polynomial are outside of the 
unit circle without actually finding them. 


Commands for Polynomials 


If alpha and beta are arrays of length p and q containing the coefficients 
of polynomials of degrees p and q whose zero degree coefficients are one, then 
the command 


gamma=INVPOLY(alpha,q,M) 


will find the first M coefficients of the reciprocal polynomial corresponding to 
that having coefficients alpha, while the command 


gamma=MULTPOLY (alpha, beta,p,q) 


will multiply the polynomials of degree p and q to find the coefficients gamma 
of the resulting polynomial of degree p+ q. 

The commands POLYROOTS and ROOTSPOLY will find the roots of a polyno- 
mial given its coefficients and find the coefficients given the roots, respectively: 


POLYROOTS(a,a0,deg,maxit,rreal,rimag,ier) 
ROOTSPOLY (rreal,rimag,deg,a). 


1.7. Describing the Distribution of Data 


TIMESLAB has three commands for studying the marginal distribution 
of data. The simplest of these is the HIST command. HIST will produce a 
histogram of a data set using either a user-specified number of intervals (bins) 
or using the number of intervals chosen by 


r 
nbins = | ot | 
3.58, ty 
where r; and sz are the range and standard deviation of the data, respectively. 


The second command one can use is INFQNT, which produces what Parzen 
(1983a) calls an informative quantile plot, that is, a plot of 


y(i) = =O versus u(i) = >, i=1,...,n, 


where m and /QR are the median and interquartile range of the data, 2,;) is 
the ith smallest z, and the values plotted on the vertical axis are truncated 
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Table 1.3. Kernels in DENSITY 


kernel Kernel Name Wit) 
1 
1 Rectangular 3 
2 Cosine (1+ cos zt) 
3 Epanechnikov 3(1 — t?) 
4 Biweight #3 (1 — t?)? 
5 Triangular 1—|é¢| 
6 Gaussian 1.012339e-"* 
4 2 3 
= — &t t US? 
7 Parzen { : 2 ' fs 
3(1 — |¢|)°, lt] > .5 


to fall between —1 and 1. Whenever an informative quantile plot:is produced, 
a reference line is placed on the plot. This line is the true quantile plot of a 
U(0,1) random variable. One can get an idea of what the informative quantile 
plot should look like for various distributions by generating a sample via the 
WN command and then calling INFQNT (see Example 1.1). 


The DENSITY Command 


TIMESLAB has the command DENSITY to form what is called a kernel 
probability density estimator. This estimator is also called a nonparametric 
density estimator since it assumes no particular functional form for the density. 
The histogram of a data set is in fact a simple form of kernel density estimator. 
It divides the range of the data into nonoverlapping intervals and counts how 
many of the data values are in each of the intervals. Then a bar graph of relative 
frequencies (counts divided by the sample size) is formed. Thus we can think 
of the height of the bar for a particular interval as the average number (or 
proportion) of the total data that are in that interval. In a more general kernel 
density estimator, the range is also divided into intervals, but these intervals 
are allowed to overlap, and the aim is to estimate the density at the center 
point of each interval. Instead of just counting the number of points in the 
interval, the method assigns a number between zero and one to each point in 
the interval with a data point at the center of the interval getting the highest 
possible value. The farther from the center that a point is, the less weight it is 
given. The function that is used to determine these “weights” is called a kernel, 
and thus the method is called the kernel method. The intervals are determined 
by specifying their center point and their width; that is, the 7th interval is 
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given by [y(i) — h, y(t) + h], where the number h is called the “bandwidth.” 
Notice that the width of an interval is not h but rather 2h. 
Formally, then, the density estimator at a point y(i) is given by 


ar aaa ee eae 


where the kernel W is a function that is nonnegative and symmetric about zero. 
Thus to form a kernel density estimate, one must specify: (1) the number of 
points at which to find the estimator, (2) the kernel to be used, and (3) the 
bandwidth to be used. In the TIMESLAB command 


DENSITY(x,n,rbins,npts,kernel,y,fy) 


the first two arguments are the data array and how many of the data points 
are to be used to find the estimator, rbins is a real scalar that determines 
the bandwidth to be used (see below), npts is an integer that determines the 
number of points at which the estimator is to be calculated, and kernel is an 
integer specifying which of seven available kernels the user wants (see Table 
1.3). Note that each of the seven kernels in Table 1.3 is defined for |t] < 1 
and is zero for |t| > 1. Each of them is in fact a probability density function 
on [—1,1] as they are positive and integrate to one. Thus the resulting density 
estimators are comparable. 

If npts is positive, then TIMESLAB will form the array y, of length npts, 
as the npts equally spaced points between the smallest and largest data points 
inclusive. If npts is negative, then the user enters in the array y the -npts 
values at which the density estimator is to be found. In either case, the DENSITY 
command will return in the array fy the estimator at the points in y. 

If the user enters a positive value for rbins, then the bandwidth that is 
used is given by 

ry 


~ Qrbins’ 


where r, is the range of the data. Specifying the bandwidth in this indirect way 
means that the user need not know the range of the data prior to issuing the 
DENSITY command. The value of rbins is very similar to the value of nbins in 
the HIST command. A large value for rbins will lead to a small value of h, and 
thus the resulting estimator will appear wiggly, just as using a large value of 
nbins in HIST will lead to a wiggly histogram. Thus typical values for rbins 
are similar to typical values for nbins, namely, in the range of four through 
twelve (see Problem C1.24). 
If the user enters a value of zero for rbins, then DENSITY will use 


Z rpn? 
Tbins = 


bay 
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Figure 1.12. Studying the Distribution of the First Difference of the Wheat 
Price Data. 


where s, is the standard deviation of the data, and gq is 1/3 for the rectan- 
gular kernel and 1/5 for the others. This translates into a bandwidth that is 
proportional to n~1/3 for the rectangular kernel and n~1/5 for the others. 

Most users of kernel density estimators agree that the choice of kernel is 
not as important as the choice of h. Choosing an optimal bandwidth is a topic 
of much discussion in the literature (see Parzen (1962a), Tapia and Thompson 
(1978), and Silverman (1986)). We will see in Chapter 3 that basically the 
optimal bandwidth in the closely related problem of spectral density estimation 
is proportional to n~!/5 for most kernels. 

Unlike HIST and INFQNT, DENSITY does not produce a plot of the density 
that it calculates. To plot it one can use the PLOT command. Several such 
densities can be superimposed on the same axes using the PLOTK command. 
This allows the user to create informative labels and scales for the plot. 

In Figure 1.12 we plot the first difference of the Beveridge wheat price 
data (the differenced values were divided by 10 so that the tic mark labels 
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on the vertical axis would be satisfactory), its histogram with the number of 
intervals chosen by the HIST command, its informative quantile plot, and three 
density estimates using the Parzen kernel for three values of rbins. Note that 
the informative quantile plot of a normally distributed sample should intersect 
the lower left and upper right corners of the plot. For the data that we are 
considering, the plot shows clearly the presence of too many large and small 
values for the data to be consistent with normality, even though the density 
estimates are clearly symmetric. See Example B.2 for another illustration of 
the use of the DENSITY command. 


1.8. Examples and Problems 


Example 1.1 | WHITE NoIsE DATA 


The macro WN.MAC finds the data plot, histogram, and informative quan- 
tile plot for white noise series having eight different marginal distributions. 
The variables n (sample size) and seed must be defined prior to starting the 
macro. If the command PSON is issued prior to invoking the macro, then the 
three graphs for each of the eight series will also be printed (the PRINTSEL 
command has to have been issued also to tell TIMESLAB what type of printer 
is attached). Inserting the PAGE command after the INFQNT will also make it 
so that the three graphs for each distribution are on separate pages. 


dos 

2a WH.MAC: macro to produce and plot white noise realizations 
Bs we} of length n from each of the 8 distributions in the 
45 WH command. 

Ceci 

Gace INPUT: n, seed (random number generator seed) 

TT Ugh 

8 55 

9 PAUSE 

10 ;start 

11 x=WH(seed,10) ;warm up generator 

12 ntype=1 ;initialize distribution counter 

13: 5 

14 ;startloop 

165 

16 x=WH(O,n,ntype) ;generate data 

17 PLOT(x,n) ;plot of time series 

18 HIST(x,n) ;histogram 

19 INFQNT(x,n) ;informative quantile plot 

20 IF(ntype.eq.8,endloop) ;check if this is last distribution 
21 ntype=ntypeti ;if not, update ntype and go back 


22 GOTO(startloop) 
23 ; 
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24 ;endloop 
25 4 


Example 1.2 | DESCRIPTIVE STATISTICS 


The macro DESCRIBE.MAC calculates and displays the basic descriptive 
statistics introduced in Chapter 1 for a univariate time series. If the macro 
were executed after having defined n=100, x=WN(12345,n), M=20, and Q=n, 
and we told TIMESLAB to obtain screen dumps of each of the graphs involved 
(from the graphics menu or by calling PSON prior to calling the macro), we 
would get the plots in Figure 1.13. 


aH DESCRIBE.MAC: macro to find and display descriptive statistics 


ah for a univariate time series. 


INPUT: x, n (data and sample size) 

Ate M (number of correlations), Q (number of frequencies 
3 in [0,1]). 

PAUSE 

;start 

PLOT (x,n) j;labels won’t be nice 
rho=CORR(x,n,M,0,1,r0,f) ;find correlations 

PLOT (rho ,M,0,M,-1,1) ;plot them 
rho=CORR(x,n,0,Q,1,r0,f) ;find sample spectral density 
PLOTSP(f ,Q,r0) ;plot it 

xx=SUBMNS(x,n,1,xbar) 

part=PARCORR(xx,n,M,rvar) ;find partials and res. vars. 


omnanw#rk OWN FP 


ee 
wmnNnaunrrh WN Ff CO 


rvar=rvar/r0O ;standardize residual variance 
LABEL(part)=’Partials (solid)’ ;prepare labels 
LABEL(rvar)=’Residual Variance (xxxx)’ ; 
PLOT2(part,rvar,M,M,1,2,0,M,-1,1) ;plot them 


Example 1.3 SCATTERPLOTS 


The macro SCATTER.MAC was used to produce the scatterplots in Figure 
1.2. The plot produced by the last line of the macro does not have nice tic 
mark labels. Thus we ran the macro, inspected the resulting plot, and then 
called PLOT again except using the seven-argument form so that we would get 
pleasing tic mark labels. We also changed the label that is placed above the 
plot. 


1. SS ed 
- O& © 
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Figure 1.13. Example Output from DESCRIBE. MAC. 


Le isis 

Dar SCATTER.MAC: macro to produce the scatterplot of a data set 
Bh Oe x (which is of length n) with itself for lag v. 
Ais 

5 INPUT: x, n, v 

6 55 

7 PAUSE 

8 ;start 

9 nmv=n-v 

10 vpi=v+1i 

11 xlag=EXTRACT(x,vpi,n) ;this is how to lag a series 

12 xx=x 

13 LABEL(xlag)=’Scatterplot for Lag #v#’ 

14 LABEL(xx)=’ 

15 mnmv=-nmv ;using a negative number of points results in scatterplot 
16 PLOT(xx,xlag,mnmv) 


Example 1.4 SUM OF FouR COSINES 


The macro COS.MAC generates four cosine curves, plots each of them and 
their sum, and then calculates, plots, and lists the periodogram of the sum. The 
periodogram is zero except for the values at the frequencies of the individual 
cosine curves. 


SEC 
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AR COS.MAC: macro to illustrate how the periodogram can 
1D decompose a series into a sum of sinusoids. 


Hie INPUT: none 

a) 

PAUSE 

;start 

PLOTON ;stay in graphics mode 

n=144 

p=<12 ,36,6,3> ;periods of the four sinusoids 
a=<2,1,6,7> ;cosine coefficients 


b=<3 ,3,1,4> ;sine coefficients 
; 


. Form individual sinusoids and the sum : 


x1=C0S(n,a[i] ,p[1]) 
yi=SIW(n,b[1] ,p[1]) 

zi=xity1 

x2=C0S(n,a[2] ,p[2]) 
y2=SIN(n,b[2] ,p[2]) 

Z2=x2+y2 

x3=C0S(n,a[3] ,p[3]) 
y3=SI(n,b[3] ,p[3]) 

Z3=x3+y3 

x4=C0S(n,a[4] ,p[4]) 
y4=SIn(n,b[4] ,p[4]) 

z4=x4ty4 

Z=z1+z2+z3+z4 

CLEAN (x1 ,y1,x2,y2,x3,y3,x4,y4) 

; Use MAXMIN to find max and min so all 5 plots will 
; be on the same scale: 

zz=<z ,z1,22,z3,z4> 
MAXMIN(zz,n,xmax, imax ,xmin, imin) 
CLEA (zz) 


; Form plotting labels : 


LABEL(z1)=’Period 12 Amplitude Squared 13’ 
LABEL(z2)=’Period 36 Amplitude Squared 10’ 
LABEL(z3)=’Period 6 Amplitude Squared 37’ 
LABEL(z4)=’Period 3 Amplitude Squared 65’ 
LABEL(z)=’Sum of four cosine curves’ 


; Plot : 

; 

BATCHOW ;no pauses between plots 
PLOTSIZE(1) ;upper left 
PLOT(zi,n,0,n,xmin, xmax) 

PLOTSIZE(2) j;lower left 


PLOT (z2,n,0,n,xmin, xmax) 
PLOTSIZE(3) ;upper right 
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55 PLOT(z3,n,0,n,xmin,xmax) 


56 PLOTSIZE(4) j;lower right 

57 BATCHOFF ;do want a pause after 4th plot 

58 PLOT(z4,n,0,n,xmin,xmax) 

59 CLS ;clear screen and get ready for next 2 plots 
60 PLOTSIZE(8) ;bottom half 

61 BATCHON ;no pause between the 2 plots 

62 PLOT(z,n,0,n,xmin,xmax) 

63 ; 

64 ; Calculate and display periodogram illustrating what 

65 ;  periodogram means : 

665 

67 rho=CORR(z,n,0,n,1,r0,per) 

68 PLOTSIZE(7) ;top half 

69 BATCHOFF ;do want pause after 2nd plot 

70 PLOTSP(per,n,rO) 

71 PLOTOFF ;Switch back to text mode for listing 


72 per=4*per/n 
73 LABEL(per)=’Periodogram’ 
74 LISTSP(per,n) ;list periodogram 


Example 1.5 TESTING FOR WHITE NOISE 


In this example we construct two graphs that are useful for determining 
whether a data set could be white noise. We will see in Sections 3.1 and 3.2 
that if 2(1),...,2(n) is a random sample from a population, then for large n: 


a) The correlations f(1),...,A(m) are independent and identically dis- 
tributed as N(0,1/n) variables. Thus there is approximately a 95 percent 
chance that an individual f(v) will be outside of +1.96/,/n. To produce simul- 
taneous confidence bands having 95 percent confidence level, we must construct 
individual intervals having level .951/". This is what is done in the first part 
of the macro. 


b) The cumulative periodogram has a 95 percent chance of falling entirely 
within the lines y = 22 + 1.36/,/q, where q = [n/2]+1. The second part of the 
macro constructs the graph of the cumulative periodogram and superimposes 
these two lines together with the line y = 2z. 


There are several features to note about WNTEST.MAC. First it uses the 
PLOTK command to do the superimposing of the plots. Second, if the user 
wants to execute only the second part of the macro (the part starting with the 
line ; per), the macro can be invoked with the command MACRO(wntest, per). 
Third, the macro creates a large number of variables that would be of no use 
outside of itself. Thus before exiting, WNTEST.MAC deletes these variables using 
the CLEAN command. In Figure 1.14 we give the result of calling the macro 
with two data sets; the first being a normal white noise series of length 100, 
and the second being the same series with a cosine of length 100, amplitude .5, 
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Correlogran for White Noise Correlogram for Cosine Plus Noise 
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Figure 1.14. Example Output from WNTEST. MAC. 


and period 4 added to it. Notice that none of the boundary lines are crossed for 
either series, but that the cumulative periodogram seems to increase rapidly 
at w = .25, something that is unusual and should be a hint that this data set 
is in fact not white noise. 


ee 
rFoOoWO MN OW WN 


OO 
nor oW WNA HW Sw LY 


A WHETEST.MAC: macro to find 95% simultaneous confidence 
sis bands for the correlogram and cumulative 
og periodogram under the hypothesis of white noise. 


2S Input: x,n,m (data, sample size, and number of correlations) 


or Ouput: rho,RO,per,cper 
PAUSE 

;start 
rho=CORR(x,n,m,0,1,r0,per) 
xx=LIWE(m,0,1) 


xx=<xx,1,m,1,m> ;xx are values on horizontal axis 
u={i+.957{1./m}}/2 ;want quantile for u={1+.95~{1/m}}/2 
zalph=DIST(z,3,1,u) ;zalph is that quantile 
lim2=zalph/{n~ .5} 

limi=-1lim2 

yy=<rho,limi,limi,lim2,lim2> 

nn=<m,2,2> 

type=<2,2,2> ;want line plots 


LABEL (xx)=<x> ;put label of data into xx 
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23 LABEL(yy)=’Correlogram and 95% Simultaneous Bands’ 

24 PLOTK(xx,yy,nn,3,type,0,m,-1,1) 

25 CLEAN(u,zalph,limi,lim2,xx,yy,nn,type) 

26: 's 

27 ;per 

28: 5 

29 yy=SUBMNS(x,n,i,xbar) 

30 q={n/2}+1 

31 FFT(yy,n,q,1,zr,zi) 

32 per={zr~2+zi-~2}/n 

33 CLEAN (yy,zr,zi,xbar) 

34 cper=CUMSP (per ,n) 

35 freq=LIWE(q,0,.5,1) ;this gets q frequencies in [0,.5] 
36 sqi=1.36/{q~ .5} 31.36 is the 95% critical value 
37 sq2=1-sqli ;the sq’s are some endpoints on graph 
38 sq3=.5*sq2 

39 sq4=.5*sqi 

40 xx=<freq,0, .5,0,8sq3,sq4,.5,0,.5,.5,.5> 

41 yy=<cper,0,1,sq1,1,0,sq2,1,1,1,0> 

42 nn=<q,2,2,2,2,2> 

43 type=<34,2,2,2,2,2> 3;all plots line plots except cum. per. 
44 LABEL (xx)=<x> ;put label of data into xx 

45 LABEL(yy)=’Cumulative Periodogram and 95% Bands’ 

46 PLOTK(xx,yy,nn,6,type) 

47 CLEAN(xx,yy,freq,nn,type,q,sqi,sq2,sq3,sq4) 


Example 1.6 MIssING DATA 


The problem of how to treat missing data in time series analysis is very 
difficult (see Problem T3.18 and Parzen (1983b) for more information about 
handling the problem). As an example of what can be done with missing data, 
the macro below replaces any element of a set of data that is zero by the 
average of the nonzero values. Note that one can enter a decimal point in a 
TIMESLAB data file for a missing value and TIMESLAB will read it as a zero. 


1 33 

3 38 MISSING.MAC: macro to fill in missing values in a data set 
Shaw with values that are equal to the mean of the 

a5 5 nonmissing values. 

5 35 

Car INPUT: x, n (data set and sample size) 

e letit 

Seer WOTE: a value is considered missing if it is identically zero. 
Oa55 

103; OUTPUT: y (same as x but has missing values filled in) 

Lise 

12 PAUSE 

13 ;start 

14 xi=EXTRACT(x,n,0,ne,ni) ;pull off the indices of nonzero values 
15 IF(ni.eq.n,end) ;go to the end if no missing values 
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Figure 1.15. The Binary Series Corresponding to a White Noise Series and 


a Pure Cosine of Period 20. 


16 xx=EXTRACT(x,xi,ni) 

17 xx=SUBMNS(xx,ni,1,xxbar,0) 
18 y=REPLACE(x,n,4,0,0,xxbar) 
19 ;end 


Example 1.7 | BINARY TIME SERIES 


;how xx is nonmissing values 
;find average of nonmissing values 
;do the replacement 


The macro below will take an input time series and produce a binary time 
series by replacing any value that is less than or equal to a by b and any value 
that is greater than c by d. If we let a=c=0, b= —1, andd =1, then we 
obtain the plots in Figure 1.15 for a white noise time series of length 100 and 
a pure cosine of length 100 with period 20. For a description of how one can 
use such a binary version of a continuous-state time series, see Problem T2.18 


and Kedem (1980), p. 34. 
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mA BINARY.MAC: macro to create a binary version of a data set 
sy x of length n. 


INPUT: x, n (original data and sample size) 

oh a, b, c, d (values less than or equal to a are 

hie given the value b, values greater than c are given 
ae the value d) 


omrnawnh WN 


i 
o 


OUTPUT: y (binary time series) 


i 
a 


PAUSE 

;start 

xi=EXTRACT(x,n,a,le,ni) ;xi’s are indices of x’s less or equal a 
y=REPLACE(x ,xi,b,n1) ;replace these elements by b 
xi=EXTRACT(x,n,c,gt,ni) ;xi’s are indices of x’s greater than c 
y=REPLACE(y ,xi,d,ni) ;replace these elements by d 
LABEL(y)=?Binary Series’ 


Example 1.8 THE SAWTOOTH FUNCTION 


A classic example of a periodic but nonsinusoidal function is the sawtooth 
function (see Figure 1.16 for an example). The macro SAWTOOTH. MAC will gen- 
erate a data set consisting of ncycs cycles of the sawtooth function where a 
cycle is defined to be 0 through amp followed by amp-1 through —amp, followed 
by -amp+1 through 0. Thus each cycle consists of 4*amp points. The variable 
amp represents the amplitude of the sawtooth. In Figure 1.16, we have used 5 
and 10 for ncycs and amp, respectively. 


a el a eo od 
oaroauwrh © WY 


Daas 

7 aS SAWTOOTH.MAC: macro to create a data set containing ncycs 
Saack cycles of a sawtooth function having amplitude 
g: ae amp. 

5 35 

6: 35 INPUT: ncycs, amp (these must both be integers) 

7 35 

8: ee OUTPUT: x, n (data set and sample size, n=4*amp*ncycs) 
9) 35 

10 PAUSE 

11 ;start 

12 api=ampt1 

13 a2=2*amp 

14 x1=LIWE(ap1,-1i,1) ;this is 0 thru amp 

15 x=x1 

16 x1=LIWE(a2,amp,-1) ;this is amp-1 thru -amp 

17 x=<x,x1> ;append to first part 

18 ma=-amp 

19 ami=amp~1 

20 x1=LINE(am1 ,ma,1) ;this is -ampt+i thru -1 

21 x=<x,x1> ; append 
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19.9 Sawtooth Function 


0.0 48.8 $0.0 120.8 168.8 200.8 


Figure 1.16. An Example of the Sawtooth Function. 


22 x1=x ;get ready to form cycles 
23 x=<0> 

24 nc=1 

25 5 

26, 581 

PS 

28 x=<x1,x> 

29 IF(nc.eq.ncycs,s2) 

30 nc=nc+1 

31 GOTO(s1) 

32 ; 

3a) 482 

34 ; 

35 n=4*ncycs 

36 n=n*amp 

37 LABEL(x)=’Sawtooth Function’ 


Example 1.9 Two METHODS OF FINDING CORRELATIONS 


In Section 1.4 we discussed the convolution and two-FFT methods of cal- 
culating sample autocorrelations. The macro below allows a user to experiment 
with the two methods and obtain precise timings for various sample sizes and 
numbers of correlations. 


AH TIMECOV.MAC: macro to compare the amount of time it takes 

a to find the first M+1 sample autocorrelations 
for a data set x of length n by the convolution 
re and two-FFT methods. 


Noawnh WN PF 


cm INPUT: x, n, m, Q (Q must be greater than or equal to n+m and 
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3 3% meet the requirements for the use of FFT) 
ON aie 

10 PAUSE 

11 ;start 

12 TIME(ti) 

13 rho=CORR(x,n,m,0,1,r0,per) ;convolution method 
14 TIME(t2) 

15 tconv=t2-ti 

16 TIME(t1) 

17 rho=CORR(x,n,m,Q,1,r0,per) 32 FFT method 

18 TIME(t2) 

19 t2ffts=t2-ti 

20 LIST(tconv,t2ffts) 


Example 1.10 | RESIDUALS FROM REGRESSION 


The macro given below was used to form Figure 1.9. 


READ(artif,y,n) ;read data 
x1=LIWE(n,1,0) ; intercept 
10 x2=LIWE(n,0,1) ;linear term 
11 x3=C0S(n,1,10) ;cosine term 
12 x=<x1,x2,x3> 

13 REG(y,x,n,3,beta,rss,t,e) 

14 PLOT(e,n,0,100,-4,4) 


Example 1.11 | Movinc AVERAGE SMOOTHER 


The macro SMOOTH. MAC was used to produce the result of using the moving 
average smoother in Figure 1.10. 


hs 

2 he ARTREG.MAC: this macro was used to construct Figure 1.9. 
cer 

4. Maa INPUT: none 

5 33 

6 PAUSE 

7 ;start 

8 

9 


si 

a SMOOTH.MAC: macro to produce a data set consisting 
Sire of the sum of a cosine (amplitude 1, 

4 3: period 40) plus N(0,1) white noise and 

SG Hs smooth it using a moving average smoother 
Go) a of 11 points. 

i 

t= Lae INPUT: none 

Tier i 

10 PAUSE 
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;start 

y=C0S(100,1,40) 

wn=WH (12345 ,100) 

y=ytwn ;y is now cosine plus white noise 
beta0=1./11. 

beta=LIWE(10, beta0,0) 

yy=FILT(y,beta,beta0,100,10) ;apply moving average smoother 


xx=EXTRACT (y ,6 ,95) ;pull off 6th through 95th values 
yy=<yy ,xx> ;prepare the plot 
xx=LINE(90,5,1) 


xx=<xx ,XX> 

LABEL(yy)=’Data (dotted) and Smoothed Data (solid)? 
LABEL(xx)=? ? 

type=<2 ,33> 

PLOTK (xx ,yy ,90,2,type,0,100,-3,3) ;do the plot 


Example 1.12 | EXPONENTIAL SMOOTHING 


The macro EXPSM.MAC determines the value of a to be used in simple 


exponential smoothing and uses it to smooth the data. 


oOmANOAA KR WN HE 


Oe Ne BRIS hl be NO ON to TE Ee Ee ee ee 
SCO MNHAAHWwNHRrRF OH WANAAH WN FO 


OF EXPSM.MAC: macro to evaluate the sum of squares of errors 
An for simple exponential smoothing for k equally 
ore spaced values between ai and a2 (inclusive) 

ns of the smoothing parameter alpha. 

a) 

m0 The sum of squares function is plotted and the 
ss minimizing value of alpha used to smooth the data. 
au A plot of the raw and smoothed data is also given. 
ae INPUT: x, n (data to be smoothed and sample size) 

sis ai, a2, k (first and last values of alpha and how 
33 many values to use) 

>”? 

PAUSE 

;start 

PROMPTOFF 

SS=LINE(k,0,0) ;Save space for sum of squares 
vals=LINE(k,a1 ,a2,1) ;vals is values to use 

i=1 ;initialize counter 

; 

;startloop 

a=vals[i] 

oa=a-1 ;get ready for difference equation 
alpha=<oa> 

w=a*x 

w=<x([i] ,w> 

xhat=ARDT(alpha,i,n,w) ;difference equation 
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31 e=x-xhat 

32 SS([i]=DOT(e,e,n) ;sum of squares 
33 LIST(a,SS[i]) 

34 IF(i.eq.k,endloop) 

35 i=it1 

36 GOTO(startloop) 

an 8 

38 ;endloop 

39 ; 

40 LABEL(SS)=’Sum of Squares Function’ 
41 LABEL(vals)=’ ” 

42 PLOT(vals,SS,k) 

43 MAXMIN(SS,k,smax, imax,smin,imin) 
44 a=vals[imin] 

45 oa=a-1 

46 alpha=<oa> 

47 w=a*x 

48 w=<x[1] ,w> 

49 xhat=ARDT(alpha,1,n,w) 

50 LABEL(xhat)=’ y 

51 PLOT2(x,xhat,n,n,2,1) 

52 PROMPTON 


am 


Computational Problems 


C1.1. Use the EDIT command to form a file containing a time series of length 
at least 50 from your area of interest and then use the DESCRIBE macro to do 
an initial analysis of the data. Do there appear to be trends or cycles in your 
data? Do your data appear to be long or short memory? If there are any 
obvious transformations that might be used on your data, try them and then 
analyze the transformed data using DESCRIBE. 


C1.2. In the WN command, TIMESLAB generates Cauchy variables by X = 
tan(7(U — $)) where U is a uniform random variable on the interval (0,1). It 
can also be shown that the ratio of two independent standard normal variables 
has the Cauchy distribution. To illustrate this, generate samples of size 200 
using both methods and use the INFQNT command to produce quantile plots 


for each sample. Do the plots look the same? 


C1.3. An example of a long-memory time series is what is called a random 
walk process (see Section 2.5.1). A simple example of such a process is to let 
a(t) be the number of heads minus the number of tails that have occurred after 
t flips of a coin. Generate a realization of length 200 from such a process by 
doing the following steps: (1) use the WN command to generate 200 observations 
from a U(0,1) distribution, (2) use the REPLACE command with iopt = 5 to 


SEC 1.8 EXAMPLES AND PROBLEMS 59 


convert these to the values z(t) = —1 if U(t) < .5 and z(t) = 1 if U(t) > .5, 
and (3) use the CUM command to find S(t) = yuae @(4) tors 1"200- Plot. 
the result. Do you see why this process is called a random walk? Find the 
correlogram of S and notice how it decays slowly to zero. 


C1.4. Use the PLOT2 command to superimpose plots of a cosine and sine of 
length 100 with amplitude 1 and period 20. How much out of phase are the two 
plots? Notice that if you were to add the two curves together, the maximum 
values in the result would have to be between those in the individual curves. 
Now add the two curves together and superimpose the plot of all three curves 
(you will have to use the PLOTK command to do this). Refer to Problem T1.3 
for more information about this phenomenon. 


C1.5. Use the WN command to generate a Gaussian white noise series of length 
100 and the CORR command to find the periodogram of the series. Use the 
PLOTSP command with third argument .001, .01, .1, 1, 10, 100, and 1000, and 
note how this moves the graph up and down. If we use the sample variance of 
the process as the third argument, then the curve will be centered around zero 
on the vertical axis. See Theorem 1.4.3 for more information about this. 


C1.6. Use the macro SAWTOOTH.MAC (see Example 1.8) to generate an ar- 
ray containing the sawtooth function of length 96 points having amplitude 2. 
Use the CORR and PLOTSP commands to produce a plot of the standardized 
log periodogram of the data. Are there harmonics in the plot? What is the 
fundamental frequency? Why? 


C1.7. Write a macro that will generate a cosine curve x of length 100 points 
having period 5 and amplitude 1 and will plot the correlogram of x for M=40. 
What is R(0)? Does this agree with what Problem T1.6 says it should be? 
Note that this problem verifies that the correlogram of a sinusoid is also a 
sinusoid. 


C1.8. Write a macro that will do the following: 


1) Generate and superimpose the plots of cosines of length n, amplitude 1, 
and periods p; and p2 for user-specified values pi and p2 of p; and po. 


2) Use the DOT command to find the sum of crossproducts SS of the two 
arrays in part (1). 


Use the macro for n = 100, and pl = 10, p2 = 20 and also pl = 10, 
p2 = 10. Do the results agree with part (a) of Theorem 1.4.1? 
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C1.9. Use the TIMECOV.MAC macro to find values of n, M, and Q for which the 
convolution method is faster, and another set for which the two-FFT method 
is faster. 


C1.10. Use the LINE command to form an array x of length 100 containing 
x[i]=i. Use the FFT command to find the discrete Fourier transform of x and 
then find the inverse discrete Fourier transform of the transform of x. Verify 
that the imaginary part of this second transform is zero and the real part is 
100 times the original array x. 


C1.11. Write a macro that will use the FILT command to form 


k 


y(t) = > axe(t — 3), te eae 


j=-k 


where aj = plil and p, n, k, and z are specified by the user of the macro. 
Notice that FILT is designed to have only coefficients with nonnegative indices 
and so you will have to be clever in how you form the beta and beta0 to be 
used by FILT. (Hint: beta0 will be a_,.) Note also that the input array will 
have to be of length n + 2k. Try out the macro for white noise input and for 
p= 8 and p=.5 with k = 1, 2,3 for each p. 


C1.12. Use the POLY command to form a time series consisting of 100 values of 
a quadratic polynomial over the interval [0,1] (you can choose the coefficients). 
Use the DIFF command twice in order to apply first differencing twice. Is the 
result a constant? 


C1.13. Form a series 


2n(t—1 
o(t) = 14.0164 cos (=), fala OO) 


and use the EXTEND command with d1i=1 and d2=10 to get the next 30 values of 
the series. Use the PLOT2 command to superimpose plots of the original data 
and the extended series. 


C1.14. Verify that the following macro gives the same result for n=100, 
amp=2.0, and per=5.0 as does the command 


x=COS(n,amp,per). 


Note that the macro uses the result of Problem T1.12 to generate a cosine 
curve. 
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133 

2 ;;  COSDE.MAC: macro to generate a cosine curve of length n having 
Smsrs amplitude amp and period per using the difference 
Aa eis equation method. 

535 

(WOR INPUT: n, amp, per 

7 33 

8 PAUSE 

9 ;start 

10 theta=2*pi/per 

11 f=line(n,0,0) 

12 £[1]=1. 

13 £[2]=COS(theta) 

14 ct=-2+f[2] 

15 alpha=<ct,1> 

16 x=ARDT(alpha,2,n,f) 

17 x=amp*x 


C1.15. Use the WNTEST.MAC macro on the first 23 and last 22 years of the 
Lake Erie levels data (use the EXTRACT command to extract the two parts of 
the series). Does there appear to be any difference in the two analyses? 


C1.16. Suppose that X is a random variable having the binomial distribution 
with parameters n and p; that is, 


f(z) =Pr(X =2) = (“)era —p)"-?, _2=0,1,...,n. 


a) Show that g(x) = log f(z) satisfies the difference equation 


g(e) ~ o(2- 1) = Ae) =log(*==** 2) >. 


(Hint: Find f(z)/f(z —1).) 


b) Find g(0) and thus use the ARDT command to write a macro that will 
find g(1),...,g(n), and thus f(0), f(1),...,f(n) for user-specified n and p. 
Note that you will have to add 1 to all of the indices as arrays in TIMESLAB 
all begin with the index 1. 


c) Check the macro by comparing its results with that of using the BINOM 
command for n=100 and p=. 4. 


Note that other discrete probability distributions can be found in a similar 
fashion. 
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C1.17. Use the REG command to find the residuals from a linear trend for the 
log of the airline data. Does this regression remove the seasonal cycle in the 
data? From these residuals, construct two data sets: (1) the 12th differences 
and (2) the result of subtracting monthly means. Use the WNTEST.MAC macro 
to analyze each of these two series. Which of the two series looks more like 
white noise? 


C1.18. Write a macro that will do the following: 


1) Generate a data set of length n made up of the sum of a linear trend having 
intercept a and slope b, a pure cosine having amplitude amp and period 
per, and a white noise series having variance sig2. 


2) Use the REG command to find the residuals from regressing the data on a 
linear trend and a cosine. 


3) Use the WNTEST.MAC macro to investigate the whiteness of these residuals. 


Try out the macro for a variety of coefficients. Find a set of coefficients 
so that the linear part predominates, another so that the sinusoid does, and 
another so that the error series does. 


C1.19. Extend the Beveridge wheat price data by 120 points using the inverse 
differencing method with d = 1. Plot the results with the horizontal and 
vertical axes ranging from 0 to 500 and 0 to 500, respectively. Use the REG 
command to fit a linear trend to the data and then extend this line by 120 
points and plot the data and the line. Use the same scaling factors as in the 
inverse differencing method. How do the two plots compare? 


C1.20. Write a macro that will calculate and plot the exponential smoothing 
weights 


6; =a(1—a)}, ae be roak th; 


for user specified values of a € (0,1) and n. The easy way to calculate the @’s 
is to find the logarithms and then use the EXP command to get their actual 
values. For what value of a do the weights decay to zero the slowest? 


C1.21. Write a macro that will successively generate white noise data of length 
100, 200, 400, 800, 1600, and 3200 and use the HIST command to plot the 
histogram of each series. Let HIST choose the number of bins each time. You 
should be able to do this in an implied loop using the IF and GOTO commands. 
Note how the number of intervals chosen by HIST increases as the sample size 
increases. The rule that is used is that the number of intervals is proportional 
to n1/3 where n is the sample size. 
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C1.22. Generate a normal white noise series of length 200 and use the DENSITY 
command with rbins=4 and the Epanechnikov kernel to find a probability 
density estimate at the 51 equally spaced points between —4 and 4 inclusive. 
Use the DIST command to evaluate the true density at the same 51 points, and 
then use the PLOTK command to superimpose plots of the estimated and true 
densities. Use a point plot (with lower triangles as the symbol at the points) 
for the density estimate and a line plot for the true density. 


C1.23. Generate a pair of independent Gaussian white noise series of length 
100. Consider the elements of these series as the real and imaginary parts 
of a complex time series of length 100. Use the POLAR command to find the 
amplitude (modulus) and phase (the arctangent of the imaginary part divided 
by the real part) of these complex numbers. Use the INFQNT command to find 
the informative quantile plot of the phases. Does it closely follow the uniform 
line on the plot? Relate this to Problem T1.4. 


C1.24. There has been some controversy as to whether the distribution of 
the Buffalo snowfall data is unimodal or trimodal. Use the DENSITY command 
with the Gaussian kernel and all combinations of 16, 32, and 48 for npts with 
rbins=3, 4, and 5 to estimate the density of the data. Do you have an opinion 
as to how many modes the distribution has? What is the bandwidth of the 
kernel when rbins=4? 


C1.25. The matrix quantities discussed in Section A.1 and calculated by the 
GS, MCHOL, and SWEEP commands are related in a variety of ways. To illustrate 
this, perform the following experiment. 


a) Generate an array X of length 40 containing a white noise series. We 
will think of X as the (10 x 4) matrix X, whose first column consists of the first 
ten elements of the array X, whose second column is the next ten elements, and 
so on. Use LISTM to display the matrix X. 


b) Use GS to find the factors Q and R in the Gram-Schmidt decomposition 
of X. Use the arrays Q and R to contain the matrices Q and R. Verify that 
X = QR by using MMULT to find the product and LIST to display it. Also 
check that Q is orthogonal by using MMULT to find Q7Q. 


c) Use MMULT to find the array XTX representing X7X, and MCHOL to find 
the Cholesky factors L and D. Verify that L= R? and D = Q7Q. 


d) Now use SWEEP to sweep X7X on its first diagonal and note that the 
second through fourth elements of the first row of the result are the same as 
the corresponding elements of R, and thus as the below-diagonal elements of 
the first column of L. Next sweep the resulting matrix on its second diagonal 
and find how the result is related to R and L. Continue this process until all 
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of the diagonals have been used. Verify that the result is the inverse of X7X 
by multiplying it by X7X and displaying the result. 


Theoretical Problems 


Y)\ 


T1.1. Show the following by induction. Alse 5 Ps is Cr 
t=\ 
a) Dots Kb aint) 
») e= n(n + = (2n + 1) 
n 2 2 
3 (n + 1) 
c) At = z 
t=1 
es (n + 1)(2n + 1)(3n? + 3n — 1) 
= 30 
Thus find an expression for the correlogram of z(t) = a+bt,t=1,...,n 


T1.2. Let f(6) = acos@ + bsin#@, and let c = Va? + 6?. Define the arctan2 
function of a and b by: 


arctan(b/a), a>0 
= arctan2(b,a) = 4 arctan(b/a) +sgn(b)n, a <0 
sen(b) = a=0, 


where sgn(b) is 1 for b > 0 and —1 for b < 0, while arctan is the usual inverse 
tangent function which takes on values only in the interval (—7/2, 7/2). 


Show that for all a and b, 
a=ccos@, b=csin¢, 


and thus 
f(@) = ccos(@ — ¢). 


(Hint: cos(#; — 02) = cos 0; cos 62 + sin 8; sin #2.) Throughout this book, when 
we write arctan we mean arctan2. 
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T1.3. Show that the sinusoid 


2 
iizik= dices == ae 
P 


has maximum values at z = (p¢/2m) + kp, for any integer k, where ¢ = 
arctan2(b/a). Show that if a > 0, then ¢/2m (or 1— ¢/27 if b < 0) is the 
fraction of a cycle that f is shifted to the right of acos2rz/p. Thus show that 
if a > 0, then f can only be shifted to the right of acos27z/p a fraction of a 
cycle contained in either the interval (0,1/4) or the interval (3/4,1). Show that 
the same thing is true for a < 0. 


T1.4. Show that if a and 6 are independent random variables each having the 
N(0,1) distribution, then c? = a? + 6? and 6 = arctan(b/a) are independent 


random variables with c? being x3 and @ being U(—z, 7). 


T1.5. Show that e*® = cos@ + isin@, and thus that e?™** = 1 for any integer 
k. 


T1.6. Let z(t) = acos2a(t—1)/p, t=1,...,n, where n = mp; that is, the 
zx’s consist of m cycles of a pure cosine with amplitude a and integer period 
pee. 

a) Show that z = 0. 


b) Show that for0 <u <n, 


~ 1 2D ree 
Riv) = a’5(1 _ ~) cos pilin 5 sin ao 
1 
where c = OR? and thus show that 

er: 
2sin — 
Pp 

p(v) =(1- ~) cos aie ee « sin bd. 


n 4 Ua P 


Thus note that the correlogram of a cosine is itself a damped cosine having 
the same period (damped because of the factor 1 — (v/n)) minus a sine having 
amplitude going to zero as n gets large. 


(Hints: If z is a complex number, then 1/z = Z/|z|?. Use real part and 
geometric series arguments as in Theorem 1.4.1. Note that cos 20 = cos? 6 — 
<9 

sin“ 6.) 
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T1.7. Suppose that 
Ye = Hj(t) + €t, t=1,....n=kd, 
where the e’s are iid N(0,07) and j(t) = mod(t — 1,d) + 1, where 


mod(I,m) =1—ml[l/ml, 


< 


and [z] denotes the greatest integer less than or equal to z. 


a) Write this model as y = Xf +, where PB = (4,...,4a)? and X is 
(n x d). 


b) Show that the least squares estimates B of B are the seasonal means 
described in Section 1.5, and that B ~ Nq(B,o7Iq). 


c) Suppose d = 7 and 
Yt =p+)> [ax cos 2 = UE by sin C= UE + €, 


fort = 1,...,n = 7m. Let y = (p, a1, a2, a3, b1,b2,63)?. Write this model as 
y = W7+€, find the least squares estimator of 7, and thus show that 


where f denotes the periodogram of the y’s. 


T1.8. Show that if we calculate the IDFT of the DFT of a set of n numbers, 
then the result is n times the original set of numbers. 


T1.9. Show that the sth difference of a set of data that is periodic with period 
s is no longer periodic. 


T1.10. Show that in the exponential smoothing model, 


-&(t +1) = az(t) + (1— @)&(t). 
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T1.11. Show that if the zeros of the characteristic polynomial for a homo- 
geneous difference equation for z are all greater than one in modulus, then 
limes cote (ee 


T1.12. Use Theorem 1.6.1 to show that the difference equation 
z(t) — 2cosw2(t — 1) + z(t — 2), bey2; 
with z(0) = 1 and z(1) = cosw has solution z(t) = cos tw. 


T1.13. Show that the zeros of the polynomials 
P . P . 
GZ)= 27 and A(z)= Soa 
9=0 j=0 


are the reciprocals of each other. 


11.14. If we form m independent confidence intervals, each having confidence 
coefficient 1 — a, what is the probability that all m of the intervals will include 
the value of the parameter that they are the confidence interval for? Thus, 
what should a be in order to have 20 confidence intervals have probability .95 
of including the true value of their parameters? 


T1.15. The informative quantile function of a continuous random variable X 
(as opposed to that for a data set) is defined by 


Q(u) — Q(5) 
TOG) ee Sowa e*(O1), 
™) = 3(9(75) = Q(25)) iow 
where Q is the quantile function of X. The quantile function is defined by, 
assuming the cdf F of X is strictly increasing, 


Q(u) =z = F(z) =Pr(X <2) =u. 


Show that the true informative quantile plot of any uniformly distributed ran- 
dom variable is the same. Show that the reference line placed on the plot 
produced by INFQNT is this informative quantile plot. 


T1.16. Show that the Parzen kernel is the probability density function of 
the average of four independent random variables, each having the uniform 
distribution on the interval [—1,1]. (Hint: Find the moment generating function 
of the average.) 


CHAPTER 2 


Probability Theory for Univariate 
Time Series 


In this chapter we consider how the basic ideas of probability theory ap- 
ply to the analysis of time series. In Section 2.2 we give the basic definition 
of covariance stationarity and introduce the theoretical autocorrelation and 
spectral density functions. In Sections 2.3 and 2.4 we discuss linear filters and 
prediction theory, and in Section 2.5 we discuss in detail the models that are 
traditionally used for data. Finally, in Section 2.6 we describe a variety of 
algorithms for univariate time series. 


2.1. Introduction 

As in many areas of statistics, our basic aim in time series analysis is 
twofold: descriptive and inferential. In Chapter 1 we considered some basic 
ways to describe time series data. To make inferences from data we will use 


the following strategy: 


1. Assume that some member of a family of models will adequately represent 
the observed behavior of a time series data set. 


2. Identify which member of the family best represents the data (model iden- 
tification). 


3. Estimate the parameters of the chosen model. 
4. Check the adequacy of the fit of the estimated model. 


5. Make statistical and scientific inferences based on the characteristics of 
the chosen model. 


Note that we will also make inferences occasionally without assuming a 
particular model for the data. We will call these nonparametric in analogy 
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299.9 2usPt Numbers, 1749-1963 
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Figure 2.1. Sunspot Numbers, 1749-1963. 


with the usual nonparametric analysis in the random sampling case. 

The aim of this chapter is to introduce both the quantities that we want 
to make inferences about (the correlogram, spectral density function, and pre- 
dictors) and the models that are usually used to allow us to make meaningful 
inferences about these quantities. 

In the typical random sampling model of elementary statistics, one’s aim 
is to make inferences about population parameters from statistics calculated 
from random samples. In time series analysis the ideas of population and 
sample are not so clear cut. We visualize that a data set z(1),..., x(n) is just 
one possible set out of many that could have been generated by some random 
mechanism that is producing data. The set of all possible realizations that 
could be observed is called the ensemble of realizations. Figure 2.1 is a time 
plot of a set of sunspot numbers observed annually from 1749 to 1963. We 
assume that annual total sunspot numbers are a random function of time— 
that is, that the observation in any one year is a random variable. Our aim 
is to make inferences about the joint behavior of these random variables when 
we have only one observation of each one. We can do this by developing ideal 
models for their joint distributions. In elementary statistics we usually need not 
consider joint behavior of observations because they are assumed independent. 


Definition. A time series is an indexed collection {X(t),t € T} of random 
variables having finite second moments; that is, E(X?(t)) < co for each element 
of the index set T. 


Usually we will consider T to be the set Z of all integers; that is, we 
will assume that the phenomenon being observed has been going on for a long 
time and will continue indefinitely. We will often refer to a time series as 
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X(t) or just X if there is no possibility of confusion. We use capital letters 
to refer to random variables and lowercase letters for particular values for the 
random variables. For the sunspot data, X(1) refers to the random variable 
representing possible sunspots in 1749, while z(1)=80.9 denotes the actual 
observed value in 1749. 


2.2. Covariance Stationary Time Series 


Traditionally, two assumptions are made about the joint behavior of the 
random variables making up a time series. The first we will label as an as- 
sumption and the second is the definition of covariance stationarity. 


Assumption. The behavior of a time series X can be adequately described 
by a knowledge of its mean value function m and covariance kernel K, that is, 


m(t) = E(X(t)), teZ 
K(s,t) = Cov(X(s), X(¢)), S462. 


If X is a Gaussian time series then this assumption is valid. We will 
see later that a wide variety of non-Gaussian time series can also be effectively 
analyzed. There is currently a great deal of effort being made in the time series 
community toward developing methods for analyzing data from processes which 
do not satisfy this assumption (see Priestley (1981, pp. 866-890), for example). 


Definition. A time series X is said to be a Gaussian time series if the joint 
distribution of any finite number of X(t)’s is multivariate normal; that is, for 
any positive integer n and any n integers t;,...,tn, we have that the joint 
distribution of X(t,),...,X(t,) is multivariate normal. 


The multivariate normal distribution plays a crucial role in the theory and 


analysis of time series. A discussion of its basic properties is given in Section 
A.A. 


Definition. A time series X is said to be covariance (or weakly) stationary if 
its mean value function is a constant, that is, E(X(t)) = y, and if there exists 
a function {R(v),v € Z} such that 


K(s,t) = Cov(X(s), X(t)) = R(t — s); 
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that is, the covariance of any pair of X’s that are the same distance apart is 
the same. The function R is called the autocovariance function of X. 


We will often drop the prefix auto in the word autocovariance. Note that 


there is another (stronger) type of stationarity, namely strict stationarity (see 
Problem T2.1). 


The Autocorrelation Function 


We usually concentrate on the correlation between two random variables 
rather than on their covariance. 


Definition. Let X be a covariance stationary time series having autocovari- 
ance function R. The autocorrelation function of X is given by 


p(v) = Corr(X(t), X(t + v)), ve Z. 


A plot of p(v) versus v for v =0,...,M is called the correlogram of X. 


Note that by the definition of the correlation of two random variables, we 
hav 
oF Cov(X(t),X(E+u)) Rv). Rv) 
Var(X(t))Var(X(t+v)) /R(0)R(0) —-R(0)’ 


and thus p(0) = 1 and p(—v) = p(v). Thus the autocorrelation function of X 
is just the autocovariance function divided by the variance R(0) of the series. 


p(v) = 


Toeplitz Matrices 
If we have a finite realization X, = (X(1),...,X(n))? from a covariance 


stationary time series X having autocovariance function R, then the covariance 
matrix of X, has the special form 


FO Meee CL mee) mers Ee PR yf) 


R(n-1) ++» ++. (2) R(1) ~—-R(0) 
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where the diagonal rows of dots indicate that each element of a particular 
diagonal is the same; that is, the (j,k)th element of the matrix is R(|j — F)). 
Such matrices are said to be symmetric Toeplitz and they have been extensively 
studied (see Grenander and Szeg6 (1958) and Example 2.1 for example). An 
(n x n) symmetric Toeplitz matrix has only n distinct elements, namely the 
elements R(0),..., R(n — 1) that are in its first row. We will use the notation 


T,, = Toepl(R(0),..., R(n — 1)) 


for such a matrix. Sometimes we will consider the correlation matrix of Xn, 
that is, 


®, = Toepl(1, p(1),..., p(n —1)). 
The TOEPL Command 
The command 
G=TOEPL(R,RO,n) 


will form the (n x n) Toeplitz matrix G with the first row given by the scalar 
RO followed by the first n-1 elements of the array R. Note that the (n x n) 
identity matrix is a Toeplitz matrix having first row (1,0,...,0) and can thus 
be formed by 


zero=LINE(n,0,0) 
I=TOEPL(zero,1,n) 


We will be primarily concerned with covariance stationary time series and 
thus will be trying to make inferences about py and the autocovariance function 
R. In Theorem 2.2.1 we summarize some of the basic facts about R, and 
introduce two functions that are mathematically equivalent to R and are of 
importance in their own right, both theoretically and in practice. Some of the 
mathematics in this theorem is quite advanced and is summarized in Section 
A.2. The interested reader can also consult Bloomfield (1976). Following the 
theorem is a discussion of its implications. 


Theorem 2.2.1 | Two SPECTRAL REPRESENTATIONS 


Let {R(v),v € Z} be the autocovariance function of the covariance sta- 
tionary time series X. Then 


a) R(v) = R(-v), ve Z. 


b) R is a positive semidefinite sequence; that is, for any positive integer 


n, any real numbers aj,...,@n, not all zero, and any n time points tj,...,tn, 
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we have 


Si Py arge - ty) > 0. 


j=1k=1 


c) SPECTRAL REPRESENTATION OF R. There exists a function F(w)we 
[0, 1], called the spectral distribution function of X , such that 


1 
R(v) = | cos 2nuwdF (w), ve Z, 
0 
where the integral is a Lebesgue-Stieltjes integral (see Section A.2.2). Further, 
F(0) = 0, F(1) = R(0), and at all continuity points of F, Fw) = R(0)-F(1- 
w). 


d) SPECTRAL REPRESENTATION OF X. There exist two uncorrelated 
stochastic processes C = {C(w),w € [0,1]} and S = {S(w),w € [0, 1]} having 


stationary uncorrelated increments (see Section A.2.3) such that X(t) can be 
written as the stochastic integral 


1 1 
Di CB his | cos 2ntwdC(w) +f sin 2rtwdS(w), 
0 0 
and for0 <w, <w2 <1, 


Var(C(w2) — C(w1)) = Var(S(w2) — S(w1)) = F(we) — F(w1). 


e) If R is absolutely summable, that is, ew |R(v)| < co, then F is ab- 
solutely continuous; that is, there exists a function f(w),w € [0, 1], symmetric 


about w = .5, called the spectral density function of X , such that 
7) 
Fw) = [ r(2)az, 
0 
dF 


(w) 
dw ’ 


fw) = 


1 
R(v) = f (w) cos 27uwdw, ve Z. 
0 


Further, f is continuous and 


co 


fw) = oF R(v) cos 27vw, w € (0, 1]. 


v=—-co 
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f) The equations relating R and f can also be written as 


1 
Rv) = | f (w)e?™" dus 


foe) 


[ee Peoelis 


v=—0o 


= R(0) +2 Ss R(v) cos 2rvw. 


v=1 


g) If f(w) > A> 0 for each w € [0,1], then R is positive definite; that is, 
the > in part (b) can be replaced by strict inequality. 


Partial Proof: Much of the theory leading to the results of this theorem is 
beyond the scope of this book, particularly parts (c) and (d). The interested 
reader can consult Priestley (1981, Chapter 4) or Fuller (1976, Chapter 4) for 
details. Part (a) follows from the fact that R(v) = Cov(X(t),X(t¢+v)) = 
Cov(X(t + v), X(t)) = R(—v), while part (b) is true because (see Theorem 
A.A4.1) 

S = Var(a’ X) > 0, 


where a? 


that 


= (aj,-..,@,) and X? = (X(t,),...,X(tn)). To see part (g), note 


non 1 
o = i otte f f(w)e?™*G-*)” dus 


f= Tk 
non 1 
>> State [erred 
»»P 0 


But this integral is zero unless 7 = k, in which case it is one. Thus 


n 
Bed, Yintt 0), 


j=1 


since we have assumed that not all of the t;’s are zero. 
The alternate expressions for R(v) and f(w) in part (f) follow from the 
facts that R(v) = R(—v) and cos(—@) = cos@, sin(—@) = —sin 6; for example, 


fone] 


co co 
So Rojee = Ss R(v) cos 2rvw — i > R(v) sin 2rvw, 


v=>—00 v=-00 v=-—-co 
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while 
nF R(v) sin 2rvw = R(0) sin(0) + SY IR(v) sin 2mvw + R(—v) sin(—27vw)] 
=0+ ys R(v)[sin 2rvw + sin(—27vw)] 


=0. 


The fact that f is symmetric about w = .5 follows from the fact that cos 27uw = 
cos 27v(1 —w). 


Implications: We are assuming that all inferences about a covariance sta- 
tionary time series X can be based on making inferences about pand R. Parts 
(c)-(e) of the theorem provide us with a function (F or f) that is mathemati- 
cally equivalent to R. These functions are important in their own right in many 
circumstances. Note that part (c) provides a harmonic analysis of the infinite 
sequence R; that is, R can be represented as a sum (actually a special kind of 
integral) of sinusoids whose coefficients are “increments” of the spectral distri- 
bution function F. Part (d) says that one possible realization {z(t),¢ € Z} can 
be thought of as a sum of infinitely many sinusoids (it is always useful to think 
of any kind of integral as a sum of a large number of terms) where within a 
particular realization the amplitudes of these sinusoids are fixed, but between 
possible realizations they vary according to a probability law. These sinusoids 
are called frequency components and we can think of f(w) as being propor- 
tional to the average value (over many realizations) of the squared amplitudes 
of the frequency component of frequency w. 


We note that F and f are analogous to a cumulative distribution func- 
tion (cdf) and probability density function (pdf) in probability theory except 
that since f integrates to R(0), they provide a distribution of variance, not 
probability. To take this analogy one step further, note that R is in the form 
of a characteristic function (it is a sequence since f is symmetric on a finite 
interval). 


We will sometimes extend f to the whole real line by making it a periodic 
function with period 1; that is, f(w+k) = f (w) for all integers k. It should 
be noted that much of the literature of time series analysis defines f to be a 
function on —7 to x or —.5 to .5. By doing a change of variables in all of the 
formulas above, we could obtain the corresponding formulas on other intervals 
(see Problem T2.8). 
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Ensemble Mean Interpretation of p and f 


Perhaps the most useful interpretation of p and f is as ensemble averages 
of the sample autocorrelation and sample spectral density functions introduced 
in Chapter 1. We will show in Chapter 3 that under very general conditions, 


E(f(w)) + f(w) and E(A(v)) > p(v) 


as the length of the sample realization goes to oo. For example, if we are 
interested in studying the EEG of a patient, we would recognize that a sample 
time series consisting of an EEG record would vary from one time to another 
in that patient’s history and we would be most interested in making inferences 
about some average behavior of the observed time series. In a situation like 
this, viewing the spectral density function as the average of the sample spectral 
density over many long realizations of EEG records is a natural thing to do. 

To further illustrate this interpretation, consider Figure 2.2, which con- 
tains the graph of the true spectral density function f of a time series X, 
together with three realizations of length 200 from X and the sample spectral 
density for each of these realizations. Finally, we have included the average 
of ten such sample spectral densities. Note how each of the sample spectral 
densities is extremely variable, but the average spectrum follows the true spec- 
tral density fairly closely. See Example 2.2 for a macro that further develops 
the interpretation of the spectral density as the ensemble average of the peri- 
odogram. 


The White Noise Process 


The simplest type of time series model is when X consists of uncorrelated 
random variables having a mean of 0 and a constant variance o?. 


Definition. A time series X is said to be a white noise process with variance 
o if 
E(X(t)) = 0, tEeZ 
o7, v=0 
(0) = Covlx(.X+ N= {5 2 


Such a process is denoted by X ~ WN(o?). 


If we define the Kronecker delta function 
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5.09. Spectra for AR(2) Process 
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Figure 2.2. Illustrating the Interpretation of the Spectral Density Function. 


we can write R(v) = 6,0? when X ~ WN(o? 


co 


fw) = 


v=-0c 


). Note that 


co 


5a R(v) cos 2muw = o? ys by cos2muw = a?; 


vu=>-0o 


that is, the spectral density of a white noise process is flat (a constant). Thus in 
analogy with the physical spectrum of white light, a sequence of uncorrelated 
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random variables (which is often used to model physical noise) is referred to as 
white noise. Processes not having constant spectral densities are often called 
colored noise. 


2.3. The Theory of Linear Filters 


We will often express one time series Y as a linear function of the values of 
another series X. We would like to be able to derive the probabilistic properties 
of Y from known properties of X. 


Definition. The time series Y is a filtered version of the time series X with 
filter coefficients {8;,j € Z} if we can write (as a limit in mean square, see 
Section A.4.4) 


Y(t) = y Bias Bate 


In Chapter 1 we defined the moving average smoother 


M 
1 
t)= SEX (f= 
Y= Yo yp Xtt-a) 
j=-M 
which is a filtering operation with coefficients 
1 lil<.M 


0, \j| > M. 


Another simple example of filtering a time series is a moving average process 
(as opposed to the moving average smoother). 


Definition. The time series Y is called a moving average process of order q if 
q 
Y(t)=D Pec(t—k), te Z, 
k=0 


where Bo = 1 and e ~ WN(o”). We will write Y ~ MA(q,f, 07) to denote 
such a series. We will also often write Y ~ MA(q) to mean that Y is a moving 


average process of order q without concern for its coefficients B or noise variance 


ms 


We have seen that if e ~ WN(o”), then R.(v) = 6,07 and f,(w) = o? 
where we have now put the subscript « on R and f to indicate to which time 
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series they correspond. The following theorem will allow us to easily find Ry 
and fy for an MA process. In fact, the theorem provides expressions for Ry 
and fy (as long as they exist) in terms of Rx and fx whenever Y is a filtered 
version of X. We can further simplify things by introducing what is called the 
covariance generating function of the autocovariance function R. 


Definition. The covariance generating function T of a covariance stationary 
time series X having autocovariance function R is the complex valued function 


foe) 


I'(z) = yi R(v)z”, zeC. 


vu=-0O 


Note that f(w) = (e727), 


Theorem 2.3.1 | UNIVARIATE FILTER THEOREM 


Suppose that X is a covariance stationary time series with autocovariance 
function Rx and spectral density function fx. Suppose that Y is a filtered 
version of X with filter coefficients B. Then assuming that the quantities 
involved exist, we have 


a) Y is also covariance stationary. 


b) The autocovariance function of Y is given by 
co 
Ry(v) = yn Re (k)Rx(v —k), ve Z, 
k=-—00 


where 


Ro(k)= > BBiey, EZ. 


j=-o 


c) The covariance generating function of Y is given by 
Ty (z) = Px(z)P(z)Pa(z7*), 


where I'x is the covariance generating function of X and 


T'g(z) = Ss Bz). 


j=—co 
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d) The spectral density function of Y is given by 


fy (w) = |h(e?"*™*) ? fx(w), 
where the function 


h(z) = Sz Buz*, zZ€E G; 


k=—00 


is called the impulse response function of the filter. 


Note that part (d) follows immediately from part (c) by substituting e?™ 
for z. We note that (eT) is called the frequency transfer function of the 
filter. We will use the Filter Theorem extensively in the sequel, particularly in 
the next section where we introduce time series models having a finite number 
of parameters. 


R and f for an MA(q) 


Let Y ~ MA(q,f,o7). Then part (a) of the Filter Theorem allows us 
to immediately conclude that Y is covariance stationary since it is a filtered 
version of white noise, which is certainly itself covariance stationary. Further 
we have 


fy (w) =o07|h(e?™”)|?, w € [0,1], 


where h(z) = )>7_, Byz* is a gth-degree complex valued polynomial. From 
Theorem 2.2.1 we also have 


q 
fy (w) = Ry(0) + a Ry (v) cos 27uw. 


Vaal 


To obtain Ry, note that 
3h" Bi Bis lol <a 
0, lv] > 9, 


and thus since R,(v) = do”, we have 


Ry(v)= > Ro(k)by-40? 


k=—00 


= Rg(v)o? 


07 I BBs, lvl <a 


0, |v] > q. 
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This is an important characterization of an MA(q) process, namely Ry(v) = 0 
for |v| > g. Note that the covariance generating function of an MA process is 
Ty (z) = oT g(z)T'g(z~1) where I'g is a qth-degree polynomial. 


The Effect of Differencing 


The Filter Theorem is also important for studying the effects of some 
transformations. For example, we can see the effect of differencing very clearly. 
Thus suppose X is a covariance stationary time series with spectral density 
function fx. Suppose Y is obtained as the dth difference of X ; that is, Y(t) = 
X(t) — X(t — d). Then Y is covariance stationary and 


fy (w) = |1—e°™* P fx(w), 


since h(z) = 1—z4. Thus fy(w) will be zero anywhere that e?7*# is one, 
namely at any w such that dw is an integer. Thus 


RAEN See he Me 


In particular, first differencing makes fy (0) = fy (1) = 0, while 12th differ- 
encing makes fy(0) = fy(;5) = --- = fy(1) = 0. Thus differencing totally 
removes frequency components of these frequencies from a time series. In fact, 
any differencing makes fy (0) = 0. We will see in Chapter 3 that the spectral 
density at zero frequency is an important quantity in making inferences about 
the mean of a time series. Because of this zeroing out of frequencies, many 
analysts suggest that great care be taken when using differencing. 


What Does the MA Smoother Do? 


Another example of using the Filter Theorem to study the effect of trans- 
formations is to consider the moving average smoother. Let X be a covariance 
stationary time series with spectral density fx and let 


M 


1 F 
Y (t) J ious pba be): 


The frequency response function of this filter is 
M 


. 1 Se 
ie) = aa FF e2tijw AS 1 Daas 
j=-M 


where the function 
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Figure 2.3. The Dirichlet Kernel. 


is called the Dirichlet kernel and is well known in many scientific areas. We 


have (see Problem T2.11) 


sin[(M + 5)27w] 
sin 7w i 


Dy(w) = 


Figure 2.3 (see Example 2.3) gives graphs of Dyy for M = 5, 10, 20, and 40. 
Note that the kernel becomes more concentrated about zero as M increases. 
Note also that the kernel is negative for certain frequency ranges and has large 
“sidelobes,” that is, secondary peaks. Thus we have 


2 
fle) = (sapzz) Wa luiPfx le) 


and Dy becomes more and more concentrated around frequency zero as M 
(the number of terms on each side of X(t) used in the average) gets large. Thus 
the moving average smoother is essentially allowing only frequency components 
of low frequency to be “passed” to Y from X. 
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Definition. If Y is a filtered version of X with frequency transfer function 
h(e?™*), then the filter is called: 


a) a low (high) pass filter if only low (high) frequency components are 
passed through the filter, that is, if fy(w) = 0 forw > uw; (w < w 1) for some 
frequency w. 


b) a bandpass filter if only frequency components in a certain interval 
(band) of frequencies are passed through the filter. 


Thus the moving average smoother is an example of a low pass filter ex- 
cept that its frequency transfer function never becomes exactly zero for high 
frequencies. We can now see why h(e2"™”) is called the frequency transfer 
function of the filter; it determines what happens to the various frequency 
components in X as a result of the filter. 


Designing Filters 


Sometimes we want to design filters, that is, determine filter coefficients 
so that an output time series Y has a certain type of spectral density. We 
illustrate this by considering the design of a bandpass filter. Suppose we want 


fx(w), w €[u—v,u+v] orw € [1—(u+v),1—(u—)] 
frw)= 
0, otherwise; 


that is, we want to determine a bandpass filter centered at frequency u and 
having bandwidth v. Recall that a spectral density is symmetric about w = Ray, 
that is, f(w) = f(1 —w) for w € [0,1], and thus we must put both intervals in 
the definition of fy. Thus we are looking for a filter having frequency response 
function 


. 1, we [u—v,u+] orw €[1—(u+v),1—(u—v)] 
h(e?™*”) = 
0, otherwise. 


This means that we must find (see Section A.2.1) 


1 
6 = | hler)e5 "da 
0 


u+u ; 1-(u—-v) : 
= / e 2tikw 7) 4. i e 7 2ttkw gy 
u-—v 1—(u+v) 


For k = 0, this gives Gy = 4v, while for k £ 0, 


i te cos 2tkusin 2rkv. 
1k 
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Thus the filter giving rise to the ideal transfer function A has infinitely many 
coefficients. In practice, we need to determine how many terms in the filter are 
needed to produce a satisfactory approximation to the transfer function. In the 
left column of graphs in Figure 2.4, we superimpose the transfer function h of 
the ideal bandpass filter for u = .25 and v = .1 and the M-term approximation 


M 
Aen (677 ss S Beer 


k=>-—M 


to h for M = 12, 24, and 48. Note that the approximation does not approach 
the ideal function in a smooth way; rather it is very wiggly and has a strange 
behavior at the points of discontinuity of h, namely at the values .15 and .35. 
This phenomenon of overshooting the function at a point of discontinuity is 
called Gibb’s phenomenon in the theory of Fourier series. Note also that the 
approximating functions are negative in places, even though the true function 
is never negative. 

The traditional method for eliminating Gibb’s phenomenon and the neg- 
ativity in approximating functions is to apply nonnegative weights to the (’s 
(the (’s are called the Fourier coefficients of h) before finding the approximat- 
ing series. This is what we have done to obtain the second column of graphs 
in Figure 2.4. We have used the Parzen weights (see Table 3.3). Note how the 
approximating functions are approaching h much more smoothly and are never 
negative. The penalty for this is that for the same M, the unweighted function 
is closer to h as measured by the integral of the squared difference between h 
and hy, than is the function using the weights. See Example 2.4 for the macro 
that generated Figure 2.4. 


The Lag Operator 


Linear filters can be succinctly represented if we introduce what is called 
the lag (or backshift) operator. 


Definition. The lag operator L operating on a time series X is defined by 
L*X(t)=X(t—k), keEZ. 


If Y is a filtered version of X we define the filter polynomial operator to be 


h(Z)i= > Bel 


k=-—0o 
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Figure 2.4. Approximations to the Ideal Bandpass Filter Transfer Function 
Using Unweighted and Weighted Fourier Coefficients. 


Thus we can write formally 
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The FILT Command 

As we saw in Section 1.5, the command 

y=FILT(x, beta, beta0,n,m) 


will form the array y by 


m 


y(i)= > Bjx(it+m—j), i=1,....n—m. 
j=0 


Thus to generate a realization of length 100 from an MA(1) process having 
8 = .5 and o? = 1, and having exponential errors, we can use 


beta=<.5> 
e=WN(0,101,3) 
x=FILT(e,beta,1,101,1) 


2.4. Time Series Prediction 


One of the major aims of time series analysis is to make use of correlation 
between and within series to predict unobserved values of a series. In this 
section we describe some of the basic theory for prediction. In Section 2.6 we 
discuss in detail several algorithms for calculating predictors for the time series 
models that we present in Section 2.5. Much of the theory of prediction is due 
to the remarkable work of Whittle (1963a). 

If we have a realization X(1),...,X(n) from a time series X, we often wish 
to find a function of the data that is close to some future variable X(n + h). 
We call n the memory or origin of the predictor and h the horizon or number 
of steps ahead being predicted. 


Definition. Let X, = (X(1),...,X(n))” be a realization of length n from a 
time series X. 


a) The best unbiased predictor of X(n+h) given X, is that function Xpp 
of X,, that has the same mean as X(n +h) and has smaller prediction error 
variance than any other unbiased function of X,,. 


b) The best unbiased linear predictor of X(n + h) given X,, is that linear 
function X,, of X, that has the same mean as X(n +h) and has smaller 
prediction error variance than any other unbiased linear function of Xy. 


c) If xh converges in mean square as n —+ oo to a random variable Xyp, 
then Xyp Is called the infinite memory, h step ahead predictor of X(n + h). 
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d) The error variances of Xnh, Xnn, and X,;, are denoted Cpe oom and 
a2 ay Tespectively. 


The following theorem is a straightforward application of the material in 
Section A.4.3 on prediction of random vectors. Before stating it we introduce 
a special matrix called the permutation matrix. 


Definition. The (n x n) matrix P,, consisting of all zeros except for ones on 
the main reverse diagonal is called the permutation matrix of order n. 


We use the permutation matrix because many of the linear combinations 
that we use are of the form 
l= at, + a2%,_1+---+ 4,21 = a’ Px, 
where a = (a1,...,0n)" and x =(z,... , Zn), since pre (post) multiplication 


of a matrix by the permutation matrix reverses the order of the rows (columns) 
of that matrix. 

There are two other important features of the permutation matrix. First, 
P?2 = In, the (n x n) identity matrix. Second, if I, is an (n x n) symmetric 
Toeplitz matrix, then (see Problem T2.13) 


[,=PalnP, and  T71=P,f;'!P,. 


With these facts in mind, we now present some of the basic facts about uni- 
variate time series prediction. 


Theorem 2.4.1 | UNIVARIATE PREDICTION 


Let X be a zero mean time series. Then 


a) The best unbiased predictor and its error variance are given by 


Xnn = E(X(n+h)|X(1),...,X(n)) 
Gan = Var(X(n+h)|X(1),...,X(n)). 


b) If X is covariance stationary with autocovariance function R, then the 
best unbiased linear predictor and its error variance are given by 


Xe Al PA 


Gan = RO) — 12 Pat rpa, 
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where X, = (X(1),...,X(n))? and the prediction coefficients 
Aphis Onn Deen 
satisfy the prediction normal equations 
Pn Ankh =Inh, 


where I’, = Toepl(R(0),...,R(n — 1)) and rp, = (R(A),..., R(h+n—1))?. 
Further, these predictors and prediction error variances can be found using 
conditional means and variances as in part (a) but for a Gaussian time series 
having the same autocovariance function as X. 


c) Let Xn(j) and G2 denote the coefficients and prediction error variances 
for the best unbiased linear one step ahead predictor. Then the ,(j) and 62 
satisfy Levinson’s recursion: 


R(j + 1) =) ey (k)R(j +1—-k) 


7 
oj 


F441 = 6F(1 a Nard +1)), 
with \3(1) = p(1) and 6? = R(0). Further, for k > 1, x(k) is equal to the 
correlation between the errors in predicting X (t) from X(t+1),...,X(t+k-1) 


and predicting X(t +k) from X(t+1),...,X(¢+k—1). Thus \,(k) is called 
the partial autocorrelation coefficient of lag k. 


Aj4i(j +1) = 


d) If X is covariance stationary and 


lim 6?, =o? >0 
noo ee : 


then X is said to be purely nondeterministic and 


i) There exists a white noise time series € having variance 02, and an 
infinite sequence of constants yo = 1, 71,2,... such that, as a limit in mean 


square, 
x)= SO Ke(t —k). 
k=0 


This is called the infinite order moving average representation of X, and the 
€’s are called the innovations of the process. 


ii) The infinite memory predictor Xnp exists and is given by, as a limit 
in mean square, 


Xnn = D> mwe(n +h—k), 
k=h 
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while 


Further, for v > 0, 


Chy = Cov(Xnnz, Xn,h+v) 


jah 
eel 
= R(v) — 02, > 40. 
7=0 


ili) A sufficient condition for X to be purely nondeterministic is that 
X have a spectral density f satisfying 


1 
Si =| log f (w)dw > —oo, 
0 


in which case 
ga Ser 


e) WoLD DECOMPOSITION. Any covariance stationary time series X can 
be written as 


X(t) = U(t)+V(0), 


where U and V are uncorrelated with each other, U is purely nondeterministic, 
and V is purely deterministic. 


Implications: Part (a) says that best predictors and their error variances 
are just conditional means and variances, while part (b) gives formulas for 
finding best linear predictors and their error variances. Note the similarity of 
the prediction normal equations to the normal equations in regression analysis, 
with R(0) playing the role of yy, I’, playing the role of XTX, andr, playing 
the role of X?y. The last part of part (b) provides an often used method for 
calculating linear predictors, namely the device of pretending that the process 
is Gaussian and then using conditional means and variances. 

Part (c) of the theorem contains the often used Levinson algorithm (see 
Levinson (1947)). This algorithm allows us to find recursively the prediction 
coefficients. We will refer to this algorithm frequently in the rest of the book. 
The last part of part (c) shows that for each memory n, the last prediction 
coefficient is indeed a partial correlation. These partials are used extensively 
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in identifying time series models (see Section 3.4). In Section 2.6 we give more 
details about part (c) (see Theorems 2.6.3 and 2.6.4). 

Part (d) of the theorem provides algorithms for finding infinite memory 
linear predictors. These are often very easy to calculate. Further, the infinite 
order MA representation of a process in terms of its innovations is often used in 
other contexts (see Theorem 2.6.8 for example). Note from part (ii) of (d) that 
the infinite memory one step ahead prediction error X(n+1)—Xpni = €(n+1), 
which is the origin of the term innovation; that is, e(n + 1) is what is left 
over after having used the infinite past of X to predict X(n+ 1). Finally, 
part (e) gives a way of decomposing any covariance stationary time series into 
two parts, one that can be perfectly predicted if we know enough of its past, 
and one part that cannot be perfectly predicted no matter how much of its 
past we know. Most of the time series models that we consider have only this 
unpredictable or nondeterministic part. 


2.5. Time Series Models 

In this section we present some of the models that have been used to 
represent data. The simplest model is the white noise process that we described 
in Section 2.3. 


2.5.1. Random Walk Processes 


The first process we discuss in this section is in fact not even covariance 
stationary, but occurs very frequently in the physical and economic sciences. 
Definition. Suppose 

X(t) = X(t— 1) + €(t), t>\1, 


where € ~ WN(o?). Then X is called a random walk process and we write 
X ~ RW(o?). 


Actually a random walk process is not fully specified until the character- 
istics of the starting value X(0) are given. We usually assume that X(0) is a 
random variable that is uncorrelated with any of the e’s. 


Theorem 2.5.1 | PROPERTIES OF RANDOM WALKS 
Suppose X ~ RW(a”) with 


E(X(0))=px, Var(X(0))=0%, Var(e(t)) = 02, Cov(X(0), €(t)) = 0. 
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5 Random Walks (n=2@8, dist=1) 


“0.2 42.0 92.0 120.0 160.8 200.0 


Figure 2.5. Five Realizations from a Gaussian Random Walk Process. 


Then 


E(X(t)) = px and Var(X (t)) = 0% + to?, eg Al 


Proof: We can write 
X(t) = X(t — 1) + €(t) 


= [X(t — 2) + e(t — 1)] + €(t) 


t 


= X(0) + So (3), 


j=l 


from which the results follow since X(0) is uncorrelated with the e’s. 
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Note that X is not covariance stationary since Var(X(t)) is not indepen- 
dent of t. Further, Var(X(t)) + 00 as t + oo. Figure 2.5 gives five realiza- 
tions of length 200 from a Gaussian random walk process; that is, X(0) and 
e(1),...,€(200) are iid N(0, 1) variables. As time progresses, the realizations get 
increasingly far apart. This is expected since Var(X(t)) is increasing linearly 
without bound as t increases. Note also that these realizations are similar in 
appearance to many price time series in economics such as stock market data. 
Finally, note that the first difference of a random walk process is a white noise 


process. 


Further aspects of the random walk are considered in Problem T2.3, and 


Example 2.5 contains the macro that produced Figure 2.5. 
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Simulating Random Walks 


A simple method for simulating random walks in TIMESLAB is 


e=WN(seed,n,dist) 
x=CUM(e,n,1) 


since the CUM command finds the sequence of partial sums of an array, and if 
this is applied to white noise, one obtains a random walk process (see Problem 
214): 


Prediction of Random Walks 
If X ~ RW(o”), then 
Xai = E[X(n + 1)]X(1),...,X(n)] 
= E[X(n) + €(n + 1)|X(1),..., X(n)] 
= E[X(n)|X(1),...,X(n)] + Efe(n + 1)|X(1),...,X(n)] 
= X(n) + Efe(n + 1)|X(1),...,X(n)]. 


If this last conditional expectation is zero, we have that X is a Martingale 
process; that is, the best predictor of X(n + 1) given the previous n values is 
just the nth value. We have not required up to now that X(0) be independent 
of the e’s (which would make X a Martingale). If X(0) and the e’s are Gaussian, 
then of course we would get that Xn1 = X(n). 


2.5.2. Harmonic Processes 


Our second example is covariance stationary but has a spectral distribution 
function that is not absolutely continuous. 


Definition. A time series X is called a harmonic process if 


M 
X(t) = }° [Aj cos 2ntw; + Bjsin2ntwj], +t Z, 
hieaiil 


where the A’s and B’s are zero mean, uncorrelated random variables having 
Var(A;) = Var(B;) = 07, and wi,...,wy are distinct frequencies in (0, .5). 
For simplicity we will assume that M = 1 and thus drop the subscripts on 


the A’s, B’s, and a?’s while leaving the subscript on w. Thus we have 


E (X(t)) = E[Acos2rtw; + Bsin 2xtw] = 0, 
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while, since E(AB) = 0 and E(A”) = E(B?) = o?, we have 


Cov(X(t), X(t + v)) = E(X(t)X(t + v)) 
= E[(Acos 2ntw, + Bsin 2rtw;) (Acos 2x(t + v)w; + Bsin 2n(t + v)w)] 


= 0” [cos 2atw; cos 2x (t + v)w1 + sin Intw, sin 2m(t + v)w4] 


= 07 cos 27 vw, 


by the trigonometric identities 
cos(@ — 2) = cosacos f+ sinasin Z and cos(—8) = cos 0. 


Thus Cov (X(t), X(t+ v)) is independent of t and X is covariance stationary 
with 


R(v) = 0? cos 2rvw}. 


Note that R(v) does not decay to zero as v - oo since it is in fact periodic. 
Thus R is not absolutely summable. Recall that absolute summability is only 
a sufficient condition for the absolute continuity of a spectral distribution func- 
tion (part (e) of Theorem 2.2.1). We can see that F has a jump discontinuity 
at w = w if we define 


0, WE [0,w4) 


o2 
FW)=4 5, we lan 1—un) 


C7. we [1 — w, 1], 


since it can be shown (see Section A.2.2) that R(v) = Np. cos 2muw dF (w). Half 
of the variance is placed at w; and half at 1—w, to satisfy the requirement 
that F(w) = R(0) — F(1 —w) at all continuity points of F. 

If we have a harmonic process with sinusoids of frequencies w,...,wag, 
then F will have jumps of size o?/2,...,02,/2 at frequencies Aer) eal’ 
w #w;, then F is differentiable with derivative zero, while at w = w;, the left 
derivative is infinite and the right derivative zero. Finally, note that a single 
realization from a harmonic process is indistinguishable from a deterministic 
sinusoid and getting more of a single realization provides no more information 
about the properties of the process. 


94 UNIVARIATE TIME SERIES CH. 2. 
2.5.3. Moving Average Processes 


In our discussion of linear filters in Section 2.3 we introduced the moving 
average process: 


q 
=) Peele he Cu ment ee 


where € ~ WN(o”), and showed that its autocovariance function R and spectral 
density function f are given by 


o atl Pr Pe+v}; |v| <q 
0, lv] > 4 
f(w) =07|h(e?™)P?, — w € [0,1], 
where the complex valued polynomial h is given by 
q 
LGA So Be ee 
k=0 
If we write the process in operator notation as 


X(t) = h(L)c(t), 


it is of interest to determine whether we can invert the operator; that is, can we 
write in any sense €(t) = (1/h(L))X(t)? The answer is given by the following 
theorem. 


Theorem 2.5.2 | INVERTIBILITY OF MA PROCESSES 


Suppose X ~ MA(q,f,o7). If the zeros of h are all greater than one in 
modulus, then 


a) f(w) > 0, w € (0, 1]. 
b) We can write as a limit in mean square, 


= Dak (t — 3), w € (0, 1], 


where the a’s are the coefficients of the polynomial 


a(z =a =D 
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and are given by ag = 1 and 


min(q,i) 


SSE) Der ezay_{\ hee ped: 
i=1 


Proof: Part (a) follows because 


|e?" |? = | cos 2mw + isin 2nw|? = cos? 2mw + sin? 2mw = 1, 


and thus the condition that h(z) # 0 unless |z| > 1 means that f (w) = 
a |h(e*) |? > 0. 

Rather than prove part (b) in general, we illustrate its proof by considering 
the simple MA(1) process X(t) = e(t) + Be(t—1). Successively substituting for 
e(t — 1), €(t — 2),... gives 


e(t) = X(t) — Be(t — 1) 
= X(t) — A[X(t — 1) — Be(t — 2)] 


= X(t) — BX (t— 1) + B[X(t— 2) — Be(t — 3)] 


K 
= Yo(-8) X(t— 3) + (BK et — K +0), 


= (—6)?* +242 +0 


as K — ov, if |B| < 1. But A(z) = 14+ Gz has the zero z = —1/ which 
is greater than one in modulus (in this case absolute value since z is real) if 
and only if |@| < 1. Note that the general expression for a; does in fact give 
a; = (—f)? as required. 


We can now formalize our definition of an invertible MA process. 
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Definition. An MA(q,B, 7) process is said to be invertible if all of the zeros 
of its characteristic polynomial h are greater than one in modulus. 


Nonidentifiability of MA Processes 


If we are given the parameters B and o? of an MA(gq) process, then we can 
calculate the corresponding autocovariances by 


rid BeBe, lvl <a 


0, |v| > q.- 


On the other hand, if we are given q and the first g + 1 autocovariances of the 
process, there is no unique set of parameters # and o? giving rise to these co- 
variances. This is called the nonidentifiability of an MA process. For example, 
for an MA(1) process, the autocovariances R(0) = 5 and R(1) = 2 could have 
arisen from o? = 1 and #, = 2 or from o? = 4 and #, = .5. For a general 
MA(q, B, o”) process, there are 27—1 other sets of parameters all leading to the 
same autocovariance function as do B and a”. These other sets of parameters 
can be determined by (1) finding the zeros of the characteristic polynomial 
corresponding to B, and then (2) constructing the other polynomials having 
combinations of these zeros and their reciprocals. Thus for an MA(2) process, 
we find the zeros z, and z2 of the characteristic polynomial, and then find 
the three polynomials having zeros z; and 1/22, 1/z; and 2, and finally 1/z, 
and 1/z2, in each case finding the value of the error variance that makes the 
autocovariance function agree with the one corresponding to the original pa- 
rameters. In Section 2.6.3 we consider methods for finding values of B and o? 
for a given autocovariance function. The values that will be found are those 
whose characteristic polynomial has all of its zeros outside the unit circle. 


Commands for MA Processes 


There are several commands that are useful for studying MA processes. 
First, MACORR and MASP calculate p and f from an MA process given its param- 
eters. The command MADT simulates data from an MA process having N(0, 1) 
errors. To get data from an MA having nonnormal errors, the WN and FILT 
commands can be combined. The CORRMA command (see Section 2.6) uses an 
algorithm developed by Wilson (1979) to find the 6’s corresponding to a given 
set of correlations. 
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Prediction for MA Processes 


In Section 2.6, we derive general expressions for the predictors and pre- 
diction error variances for an MA process. These expressions will be functions 
of the elements of the modified Cholesky decomposition (see Section A.1.1) 
of the covariance matrix of the MA process. A distinguishing feature of this 
covariance matrix is that it is banded; that is, its elements outside of a band 
formed by its main diagonal and the q diagonals on either side are zero. This 
leads to simple forms for the predictors. For now we note that for a Gaussian 
MA process, 


Xnk = E(X(n+h)|Xn) 
= D0 PE (c(n + h — k)|X,) 


=0, h>q, 


since ¢(t) is independent of X(s) for t > s. Thus by the device described in 
part (b) of Theorem 2.4.1, we can write the following recursion for the best 
linear predictors: 


A See Aad, es Lilt 
Xx, 


0, h>q. 


Partial Autocorrelations for MA Processes 


In Theorem 2.4.1 we saw that the partial autocorrelation of lag v of a time 
series X is the last coefficient A,(v) in the best v step ahead linear predictor. 
In Section 2.5.4 we will describe a simple method for calculating these partial 
autocorrelations. For now we note that unlike the ordinary autocorrelations 
which become identically zero for lags greater than the order q, the partial au- 
tocorrelations of an MA process decay to zero exponentially (see the examples 
at the end of this section). 


MA Spectra and Trigonometric Polynomials 


Before leaving MA processes, we note that since R(v) = 0 for |v| > q, we 
can write from part (f) of Theorem 2.2.1, 


q 
f(w) = R(0) +2 pis R(v) cos 27uw. 
v=1 


Now we can also write cos 2tvw as a polynomial of degree v in cos 27w since 
for v > 2, we have the important trigonometric identity 


cos v9 = 2cos @ cos(v — 1)6 — cos(v — 2)0 


>] 
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which if used recursively, ultimately expresses cos v@ as a polynomial in cos @. 
Thus the spectral density of an MA(q) can be written as a qth-degree trigono- 
metric polynomial. The above identity is also important in other contexts in 
time series analysis. If we write z(v) = cos v#, we have 


z(v) — 2cos@ z(v — 1) + z(v — 2) =0, as 


with z(0) = 1 and z(1) = cos@, which is a second-order difference equation 
with initial conditions z(0) and z(1). This equation has solution z(v) = cos v@ 
(see Problem T1.7). 


Examples of MA Processes 
In this section we have seen: 


1. The autocorrelation function for an MA(q) process is identically zero for 
lags greater than the order q. 


2. The partial autocorrelation function decays to zero as v increases. 


3. The spectral density function of an MA(q) process is a gth-degree trigono- 
metric polynomial. Thus for small values of q, it is difficult for the spectral 
density to have sharp peaks. 


In Figure 2.6 we display the correlogram, partial correlogram, spectral 
density, and one realization of length 200 from each of two MA processes. The 
two processes are: 


Model 1: X(t) = €(t) — 0.70e(t—) — 0.10e(t — 2) + 0.60e(t — 3) 
Model 2: X(t) = e(t) + 0.80e(t — 4); 


that is, the first process is an MA(3) process while the second is a (subset) 
MA(4). In both cases, the correlogram is zero for lags greater than the order 
of the process, while the partial correlogram is not. The partials for the subset 
model are zero except for lags that are multiples of four. See Example 2.7 for 
the macro that generated this figure. 


2.5.4. Autoregressive Processes 
Perhaps the most often used time series model in practice is the autore- 
gressive process. Intuitively, X is an autoregressive process of order p with 


coefficients a1,...,@p and noise variance o? if it can be transformed to white 
noise by a filter of length p; that is, 


X(t) +aiX(t—1)+---+a,X(t-p)=et), te Z, 
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Figure 2.6. Correlogram, Partial Correlogram, Spectral Density, and a Real- 
ization of Length 200 for Each of Two MA Processes. 


where € ~ WN(o?). One appeal of this process is that it is in the form of a 


regression model 


X(t) =—dyX(t =) < 


"+= apX(t ~p) + e(t), 


with past (or “lagged”) values of X as the independent variables in the regres- 


sion. 
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It is important to note that the autoregressive process is not defined ex- 
plicitly, but rather as the solution to a stochastic difference equation, that is, a 
difference equation whose terms are random variables. The difficulty with this 
is in determining whether there is in fact a covariance stationary time series 
that satisfies such an equation. To illustrate this, suppose that X satisfies the 
difference equation with p = 1: 


X(t) + aX (t — 1) = €(t). 
Successively substituting for X(t — 1), X(t — 2),... gives 


K-1 


X(t) — D5 (-ae(t — j) = —(a)* X(t — f). 


j=0 


If |a| < 1, then taking the limit of the expected value of the square of both 
sides of this results in 


X(t) = > Bje(t — 5), 
j=0 


where 2; = (—a)’. 


1 1 
Now we could also write —X (t+ 1) + X(t) = 
a 


ait +1), which gives upon 


successive substitution 
1 
xi) = pee 1) + e€(t + 1)] 


z —=[-=(x¢t + 2) +¢(t+ 2)) + (t+) 


which, if |a| > 1, gives as a limit in mean square 
foe} 
X(t) = D5 Bet +5), 
g=1 


where 8; = (—1/a)). If |a| = 1 we are not able to do this process as neither 
(—a)* nor (—1/a)* goes to zero. 

These arguments can be extended to p > 1 where instead of the location 
of |a| determining whether to write X(t) as a function of past and present or 
future ¢’s, the location of the zeros of the complex valued polynomial 


P 
G2) = > az, Go = 1; 
j=0 
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determines the representation. The following theorem summarizes the basic 
results. 


Theorem 2.5.3 | PROPERTIES OF AUTOREGRESSIVE PROCESSES 


Let ag = 1 and aj,...,ap be a set of real constants, and define the com- 
plex valued polynomial (called the characteristic polynomial of the stochastic 
difference equation) g(z) = )°*_, ajz7. Let ¢ ~~ WN(o?). Then 


a) If none of the zeros of g are equal to one in modulus, then 


X()= DY) Bet-3), tz, 


j=-0 


exists as a limit in mean square where the (’s are the coefficients of the poly- 
nomial 


This representation is called the (doubly) infinite order moving average repre- 
sentation of X. Further, X satisfies the autoregressive difference equation, is 
covariance stationary, has spectral density 


f(w) = 0? 


= 2 Qriw\|2 
[g(e2"*) 2 o*|h(e™"™)| 


and has autocovariance function R satisfying 


P 
So aj RG - v) = 078», ve Z. 
j=0 


b) If the zeros of g are all Jess than one in modulus, then B; = 0 for j > 0; 
that is, X(t) can be expressed as a function of €’s at only future times. 


c) If the zeros of g are all greater than one in modulus, then 8; = 0 for 
j < 0; that is, X(t) can be expressed as a function of €’s at only the present 
and past times. Thus the equations relating the R and the a’s can be written 


as 
P 


aU) Shelia wi Ud 


a0) 


which are called the Yule-Walker equations. 
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d) If any of the zeros of g are equal to one in modulus, then there is 
no MA(oo) representation of X, and in fact there is no covariance stationary 
solution to the autoregressive difference equation. However, if one specifies the 
distribution of any p consecutive X ’s, one can generate a (nonstationary) time 
series satisfying the difference equation and agreeing with the specified initial 
conditions. 


Proof: The results in parts (a)—(c) about the coefficients of 1/g(z) follow from 
properties of Laurent expansions of analytic functions (see Ahlfors (1953)). 
The fact that the infinite order MA process satisfies the difference equation is 
verified by some simple algebra. We can then use the Filter Theorem to say 
that X is covariance stationary and that the spectral densities f, and f of «€ 
and X are related by 


o” = fe(w) = |g(e?™)/?F(w), 


from which we obtain 


o 


Ig (e2"*~) 2" 


To verify the last equation in part (a), we multiply both sides of the autore- 
gressive difference equation by 


fw) = 


X(t—v)= B.e(t — v — k) 


k=—0o 


and take expected values of both sides, which gives 


Sang» =" («6 E autt—»-W)) 


7=0 k=—0co 


ao? yo Brok+u 


k=-co 
2 
=o°p_y. 


Part (d) is just a generalization of the random walk process. Given initial 
conditions, we can generate all the rest of the X’s from the e’s via the difference 
equation. 


From this theorem we can see that as long as none of the zeros of g are 
on the unit circle (that is, equal to one in modulus), the infinite order mov- 
ing average process provides a covariance stationary solution to the stochastic 
difference equation and has the properties given by the theorem. If all of the 
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zeros of g are outside the unit circle, then Problem T2.15 describes a method 
for explicitly defining a stationary solution. 

Since in practice we do not observe the future, we would like to define 
an autoregressive process so that X(t) is only a function of {e(s),s < t}. We 
formalize this in the following definition. 


Definition. A time series X is said to be an autoregressive process of order 
p with coefficients a = (a1,...,a@,)" and noise variance a? if X satisfies the 
stochastic difference equation 


X(t) +a, X(t—1)+---+a,X(t —p) = e(t) 


where the zeros of g(z) = )*_, a;z) are all greater than one in modulus. We 
denote such a time series by X ~ AR(p, a, 07). 


Recall for an MA(q) process that R(v) = 0 for |v| > q; that is, as soon 
as the lag v gets larger than the order of the process, X(t) and X(t + v) are 
no longer correlated. The autoregressive process has a correlation function 
that often gives a more realistic description of how X(t) and X(t + v) become 
uncorrelated as v increases. We will illustrate this by first considering the 
AR(1) and AR(2) cases and then presenting a general result about the behavior 
of the autocorrelation function in Theorem 2.5.4. For an AR(1) process having 
coefficient a and error variance o”, we can write the Yule-Walker equations for 
v=0andv=1as 


R(0) + a@R(1) = 0? 
R(1) + aR(0) = 0, 


since R(1) = R(-1). From the second equation we have R(1) = —aR(0), 
which when substituted into the first equation gives 


2 


womens 


R(0) 
From the Yule-Walker equations for v > 0, we have 
R(v) = —aR(v — 1) = (—a)?R(v — 2) = --- = (-a)"R(0), 


which gives 
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Finally, since R(v) = R(—v), we have 


Saeed 
R(x =F pny =(-a)!", vez. 


Note that a sequence {a(v),v > 0} is said to decay exponentially to zero if we 
can write 


la(v)| < cd”, 


where |6| < 1. Thus the autocovariance function (and hence the autocorrelation 
function) of an AR(1) process decays exponentially to zero. The rate of decay 
depends on how close a is to one. 

For an AR(2) process, we have for v > 2, 


R(v) + a, R(v — 2) + agR(v — 2) = 0, 


which is a second order homogeneous difference equation. From Theorem 1.6.1, 
we know that R(v), which is the solution to the difference equation, can only 
be one of three forms depending on the nature of the zeros z; and z of the 
polynomial z?+a1z+a. If they are real and distinct, then R(v) = 812¥+22%; 
if they are real and equal, then R(v) = (G1 + Got)z{; and if they are complex 
conjugate pairs, we have R(v) = ycos(v@ + 6)|z1|". In all three cases, R will 
decay to zero exponentially since the zeros of z? +a 1z+ «2 will be less than one 
in modulus since they are the reciprocals of the zeros of 1+a,z+ a 2z? which are 
outside the unit circle. Here the rate of decay depends on how close to the unit 
circle is the zero that is closer to it. If the zeros are complex, the autocorrelation 
function will appear sinusoidal as well as decaying exponentially. 

The next theorem shows that this exponential decay of p is true for every 
AR order. 


Theorem 2.5.4 | EXPONENTIAL DECAY OF AR CORRELATIONS 


Suppose X ~ AR(p,a@,o”) and let R be the autocovariance function of X. 
Let z1,...,2p be the zeros of g(z). Then 


1R(0)| <x (max =) 


+ 


for some constant Kk > 0. 


For details on the proof of this theorem, see Anderson (1971), p. 175. 
Note that the theorem says that R(v) is an exponentially decaying function of 
the reciprocal of the modulus of the zero of g that is closest to the unit circle. 
If a zero is close to the unit circle, R(v) may decay very slowly. 
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Autoregressive processes are very popular also because as we will see in 
Chapter 3, the estimation of their parameters and the forecasting of future val- 
ues are rather easy from a computational point of view. Further, the following 
theorem (see Fuller (1976), p. 150) shows that a wide variety of covariance 
stationary time series can be arbitrarily well approximated by an AR process. 


Theorem 2.5.5 | AUTOREGRESSIVE APPROXIMATION 


Let X be a covariance stationary time series having spectral density fx. 
Then for any 6 > 0, there exists a time series Y having spectral density function 
fy such that 


lfx() — frw)l<6, we [0,1], 
while Y satisfies . 
> Y(t — 5) = (2), 
j=0 


where p is a finite integer, aj = 1, the zeros of DL y=0 a;z) are all outside the 
unit circle, and € ~ WN(o”). 


Prediction for AR Processes 


Prediction is very simple for AR processes as we can write for a Gaussian 
AR(p) process 


Xnn = E(X(n+h)|Xn) 


= E(e(n+h)|X,) — dU ajE(X(n +h-—1)|Xn) 


P ~ 
=~) a5Xn nj 
g=2 


since ¢(t) is independent of X(s) for t > s. Note that for j > h, none; = 
X(n+h—j) since then X(n+h—j) is an element of the set being conditioned 


upon. Thus the AR predictors satisfy the homogeneous difference equation 


Further 
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where the (’s are the coefficients of the MA(oo) representation of X. 

From these facts, we also obtain an important fact about the partial au- 
tocorrelations 9 of an AR(p) process, namely that for any lag v greater than 
the order p, we have that 0, = 0. 


Commands for AR Processes 


TIMESLAB has several commands for studying AR processes. The al- 
gorithms used for several of the commands are described in detail in Section 
2.6. First, ARDT will simulate a realization from a Gaussian AR process. To 
generate a realization of length n from an AR process having non-Gaussian 
errors, one can use code of the following form: 


n2=24n 
ep=line(p,0,0) 
e=WN(seed,n2,dist) 
e=<ep,e> 
x=ARDT(alpha,p,n2,e) 
npi=n+1 
=EXTRACT(x,npi,n2) 


where dist is the code for the distribution desired for the errors. Note that 
this uses zeros as starting values in the difference equation and then lets the 
recursion run for twice the length of the desired data set, and extracts the 
second half of the realization to be used as the data set. The effect of the 
starting values should be eliminated in this way. 

The ARCORR and CORRAR commands find correlations from parameters and 
vice versa. The ARSP command calculates the spectral density function of an 
AR process from its parameters, while ARPART and PARTAR do the transfor- 
mations from AR parameters to partial autocorrelations and back again. Note 
that ARDT, ARCORR, and ARPART all check whether the zeros of the characteristic 
polynomial are all outside the unit circle. 

The command DTAR is a very general command that can be used to identify 
AR orders and estimate AR parameters by a variety of methods. See Section 
3.5 for the methods that are used in the command. 


Examples of AR Processes 
We have now seen that for an AR(p) process: 
1. The autocorrelation function satisfies a difference equation of order p 
(namely the Yule-Walker Equations) and thus, since the zeros of the char- 


acteristic polynomial are all outside of the unit circle, the correlogram 
decays exponentially to zero. 
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2. The partial autocorrelation function is identically zero for lags greater than 
pt 


3. The spectral density is the reciprocal of a pth degree trigonometric poly- 
nomial. This means that it is capable of having very sharp peaks. 


In Figure 2.7 we give the correlogram, partial correlogram, spectral density, 
and a realization of length 200 for each of the two AR models: 


Model 1: X(t) — 0.80X(t — 1) + 0.40X(t — 2) = €() 


Model 2: X(t) — 0.90X(t — 1) + 0.70X(t — 2) = €(t). 


Note how the qualitative features of the data correspond to the spectra and 
correlations. In particular, notice that the correlograms are of two types. In 
Series 1, the correlogram decays relatively rapidly to 0, while in Series 2, the 
correlogram follows a sinusoidal decay. See Example 2.7 for the macro that 
generates the graphs in this figure. 


Periodically Correlated AR Processes 


A common occurrence in seasonal data is that each X (¢) is linearly related 
to previous values of X, but that the relationship (as expressed by the number 
of previous values or their coefficients) varies from one season to another. A 
model that has been used to reflect this possibility is the periodic autoregressive 
model (see Jones and Brelsford (1967), Pagano (1978), and Newton (1982)). 
In the definition that follows we will assume that the X’s have been arranged 
so that X(1) corresponds to the first of d “seasons.” 


Definition. A time series X is said to be a periodic autoregressive process of 


period d, orders p,,..., pa, coefficients a, (j), and residual variances eee ae 
if 

Pr 

Yar(X(t-j =), teEZ, 

43=0 


where k = mod(t — 1, d) + 1, and the e’s are zero mean, uncorrelated random 
variables with Var(e(t)) = 07. 
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Figure 2.7. Correlogram, Partial Correlogram, Spectral Density, and a Real- 
ization of Length 200 for Each of Two AR Processes. 


2.5.5. Autoregressive-Moving Average Processes 


In the previous two sections we studied MA and AR processes. In this 
section we consider the autoregressive-moving average (ARMA) process which 
in a variety of ways is a combination of the AR and MA processes. 
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Definition. A time series X is said to be an autoregressive-moving average 
process of orders p and q, coefficients a = (aj,... CAE and B = (f1,...,8,)", 
and noise variance o? if X satisfies 


P 


do aj X(t- 3) =) > fe(t—k), ted, 
k=0 


7=0 


where € ~ WN(o”) and the zeros of the complex polynomial g(z) = Byer a;z4 
are all outside the unit circle. We denote such a time series by 


X ~ ARMA(p, q,@, 8,07). 


If the zeros of the complex polynomial h(z) = peo Bez* are all outside the 
unit circle, then we say that X is invertible. 


The operators g(L) and h(L) are called the AR and MA operators, respec- 
tively. Using arguments similar to those given in the previous two sections, we 
can show that an ARMA process has the MA(co) representation 


co 
X() => melt-h), te, 
k=0 
where the 74’s are the coefficients of the polynomial h(z)/g(z), while if X is 
invertible, it has the AR(oo) representation 


Yo ayX(t-j=c(t), teEZ, 


y=0. 


where the aj’s are the coefficients of the polynomial g(z)/h(z). Thus we are 
able to justify the idea of writing g(L)X(t) = h(L)e(t) as 


X(t) = FLO 
g(L) id 
Fay * (= (0: 


In the next theorem, we express the autocovariance and spectral density func- 
tions of an ARMA process in terms of its parameters. 
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Theorem 2.5.6 | R AND f FOR AN ARMA PROcEss 


dhe 
X ~ ARMA(p,q,@, 8,07), 


then 


alh(een) 
a) F(a) = 0" (garta) 


b) The autocovariances R satisfy 


q 


P 
>a; R(j — v) = )> &Rx.(v—k), v>0 
g=0 


k=0 
=0, v>q, 
where the cross-covariances Rx, are given by 


0, v>0 
Rx-(v) = Cov(X(t), e(t+ v)) = 


ea. op v <0 
and the y,’s are the coefficients of the MA(oo) representation of X. 


c) The covariance generating function x of X is given by 


where h and g are the characteristic polynomials for the MA and AR parts of 
the model, respectively. 


Proof: To prove part (a), define the time series Y by Y(t) = g(L)X(t) = 
h(L)e(t), and note that since Y is a filtered version of €, we know that it is 
covariance stationary and we can write fy(w) = o?|h(e?”)|?. Further, since 
the zeros of g are all outside the unit circle, we know that X could be written 
as a filtered version of the covariance stationary series Y and thus is itself 
stationary, and fy(w) = |g(e?"™)|?fx(w). Equating the two expressions for 
fy (w) and solving for fx(w) give part (a). 

Multiplying both sides of the ARMA difference equation by X(t — v) and 
taking expected values give 


P 


S> ajE(X(t — j) X(t - »)) = So Ble t — k)X(t — v)) 
k=0 


j=0 
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Table 2.1. Nature of R and f for ARMA Processes 
eee 


Process R(v) or p(v) f(w) 
MA(q) 0 for |v| > ¢ qth-degree trigonometric 
polynomial 
AR(p) exponential decay reciprocal of pth-degree 
for v>0 trigonometric polynomial 
ARMA(p, q) exponential decay (q, p)th-degree rational 
for v >q trigonometric polynomial 


or 
P q 
5 RG - v) = Se Rxe(v — bk). 
7=0 k=0 


Since X(t) can be expressed in terms of ¢(s) for s < t, we have that X (t) is 
uncorrelated with future e’s; that is, Rx<(v) = 0 forv >0. For v < 0, we have 


Rxe(v) = Yn E(€(t — ke(t + »)), 
k=0 


and all of these expectations are zero except when k = —v. 


Thus R satisfies a homogeneous difference equation of order p for v > q. 
The equations for v = q+ 1,...,q+ > are often called the high order Yule- 
Walker equations. We can now summarize the nature of R (or p) and f for 
AR, MA, and ARMA processes as in Table 2.1. 


Prediction for ARMA Processes 


In Section 2.6 we derive algorithms for finding predictors and their error 
variances for ARMA processes from two points of view. In Section 3.4.2 we 
describe a method for finding approximate predictors and variances. 


Commands for ARMA Processes 


The ARMADT command generates a realization from a Gaussian ARMA 
process, while to get a realization from an ARMA process having non-Gaussian 
errors, one can use the ARDT command in a similar fashion to generating non- 
Gaussian AR data except that the forcing term in the difference equation is a 
realization from a non-Gaussian MA process rather than a non-Gaussian WN 
process (see Problem C2.8). 

The ARMASP command calculates the spectral density of an ARMA process, 
and the ARMACORR and CORRARMA commands find correlations from parameters 
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and vice versa. The ARMAPRED command finds predictors and prediction error 
variances for ARMA data. The algorithms for these last three commands 
are described in Section 2.6. To estimate parameters of ARMA schemes, we 
describe in Chapter 3 the ARMASEL, DTARMA, and SEASEST commands. 


Examples of ARMA Processes 


As we have seen, the ARMA process is a combination of the AR and MA 
processes. The basic characteristics of an ARMA model are: 


1. Neither the correlogram nor the partial correlogram is identically zero past 
some lag; rather they each decay exponentially past some lag. 


2. The spectral density function is the ratio of a qgth-degree trigonometric 
polynomial to a pth-degree trigonometric polynomial. Thus it can have 
sharp peaks and/or sharp troughs. 


In Figure 2.8 we give the correlogram, partial correlogram, spectral density, 
and a realization of length 200 from each of the two ARMA models: 


ily X (t) — 1.20X(t — 1) + 0.78X (t — 2) = e(t) + 0.45€(¢ — 1) + 0.39e(t — 2) 
2 X (t) — 0.90.X(t — 1) = e(t) + 0.80e(t — 4). 

That is, Model 1 is an ARMA(2,2), while Model 2 is an ARMA(1,4) where 
the first three MA coefficients are zero. In each of these models, both the 
correlogram and the partial correlogram exhibit an exponential decay, thus 


ruling out a pure AR or pure MA model. See Example 2.7 for the macro that 
generated this figure. 


2.5.6. Subset and Multiplicative Subset ARMA Processes 


A natural extension of the ARMA model is the case where only a few of 
its coefficients are nonzero; that is, the subset ARMA models which we write 


as F ; 
(>: au) A(t) = (>: nat* | e(t), 
k=0 m=0 


where uj; < --- < up = pand vy, <--- < vg = q are called the AR and MA 
lags, respectively. We can go a step further and define a multiplicative subset 
ARMA process as a combination of a “full” ARMA and subset ARMA by 


P BE q Q 
(3: a) (>: aim ) X(t) = (szau') (5s rns €(t). 
4=0 k=0 1=0 m=0 
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Figure 2.8. Correlogram, Partial Correlogram, Spectral Density, and a Real- 
ization of Length 200 from Two ARMA Processes. 


Such models will be important in Chapter 3. Note that as long as the zeros of 
each of the four polynomials involved in the model are all outside the unit circle, 
then we can express the multiplicative subset ARMA model as an ordinary 
ARMA model of orders p+ up and q + vg, and use all of the results that we 


have obtained for ARMA processes. 
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2.5.7. ARIMA Processes 


Many of the processes in economics and business are nonstationary but 
can be transformed to stationarity by differencing. This fact has led to the 
popularity of the so-called Box-Jenkins method of time series analysis (see 
Box and Jenkins (1970)). The method employs extensively what is called an 
ARIMA model. 


Definition. A time series X is called an autoregressive integrated moving 
average process of orders (p,d,q) if the series Z(t) = (1 — L)4¢X(t) is an 
ARMA(p,q) process. We denote such a process by 


X ~ ARIMA(p, d,q, @, B, a”). 


The simplest example of such a model is the random walk process where 
d=1landp=q=0. Note that if we let g and h denote the AR and MA 
operators for the ARMA part of the model, then we have that X satisfies 


g(L)(1 — L)4X(t) = h(L)c(t). 


The difficulty with ARIMA processes is in expressing exactly what is meant 
by this equation. As we have discussed for the random walk model, there is 
no MA(oo) representation for X. However, we can visualize specifying some 
initial conditions and then generating all of the rest of the X’s by the difference 
equation. 


As a simple example, let p= 1, g= 1, and d= 2. This gives 
(1 + oL)(1 — L)?X(t) = (1 + BL)e(t), 
that is, 
X(t) + (a — 2)X(t — 1) + (1 — 2a) X(t — 2) + a X(t — 3) = €(t) + Be(t — 1). 


If we specify any three consecutive X’s, we can then construct an infinite series 
of future X’s in terms of a WN(a?) process € by this difference equation which 
can also be solved to obtain an expression for past X’s. Note that the variance 
of these X’s grows without bound as ¢ increases as in the random walk case. 
This sort of specification of initial conditions is implicit in any definition of 
an ARIMA model and should be kept in mind. For example, the joint pdf of 
X(1),...,X(n) for an ARIMA model depends on what initial conditions one 
is assuming (see Ansley and Kohn (1985)). 
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Figure 2.9. The Airline Data and Successive Transformations. 


Multiplicative Subset Seasonal ARIMA Models 


A general model of the form 
g(L)G(L)(1 — L)#(1 — L°)? X(t) = A(L)H(L)e(t), 


where g,G,h, and H are the four operators in the multiplicative subset ARMA 
model, provides a general framework within which many seasonal time series 
can be analyzed. If the AR and MA lags are multiples of the seasonality factor 
S, then this model is the one which is used in the Box-Jenkins forecasting 
method (see Section 3.6). The most well known example of this kind of model 
is the international airline passengers data (Series VI in Section 1.1). In Figure 
2.9 we give the original data and then the results of successively taking natural 
logs, first differences of the logs, and then 12th differences of the first difference 
of the log. Note how the log transform seems to stabilize the variance, the first 
difference removes the linear trend, and the 12th difference removes the seasonal 
cycle. Box and Jenkins suggest that a model with p= 0,P =0/q¢=1, and 
Q = 1 with v; = 12 fits the first and 12th differences of the log of the airline 
data. 
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2.5.8. ARMA Models and Inverse Autocovariances 
Let X be a covariance stationary time series with autocovariance function 


Rand spectral density f where f is bounded above and below by positive, finite 
constants. Then we can make the following definition (see Cleveland (1972)). 


Definition. The inverse spectral density function fi, inverse autocovariance 
function Ri, and inverse autocorrelation function pi of a covariance stationary 
time series having spectral density function f are given by 


w € [0,1] 


1 

Ri(v) = st filw)e2"""dw, vez 
0 
Ri 


i) — Ri(v) 
pi(v) = Rilo)” 


veEZ. 


These functions are useful for ARMA models as indicated by the following 
theorem. 


Theorem 2.5.7 | INVERSE PARAMETERS FOR ARMA MODELS 


Let X ~ AR(p,a,o7), Y be an invertible MA(q,B,c7), and Z be an 
invertible ARMA(p, q,a, 8,0”). Then 


a) The inverse spectral density fix and autocovariance function Rix of 
X are given by 


fix(o) = lol)? 


P 
= ds Rix (v) cos 2ruw 


v=-p 


P 
= Rix(0)+2 yf Rix (v) cos 2rvw, 
v=1 


where 1 
> eyera OjAj4Jo}, lvl <P 


0, |u| > p; 
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that is, fix and Rix are the spectral density and autocovariance functions of 
an MA(p, a, 1/07) process. 


b) The inverse spectral density fiy and inverse autocovariance function 
Riy of Y satisfy 


1 1 
o2 |h(e2**~) |2 


fiy(w) = 


q 
>> Be Riy (k — v) = bua, v > 0; 


k=0 


that is, fiy and Riy are the spectral density and autocovariance functions of 
an AR(q,8,1/07) process. 


c) fiz and Riz are the spectral density and autocovariance functions of 
an ARMA(q,p, B, a, 1/07) process. 


We will use the duality between AR and MA processes described by this 
theorem to aid in model identification in Section 3.4. 


2.5.9. The General Linear Process 


In Chapter 3 we will describe the statistical properties of estimators of R 
and f for a time series X under as general a set of assumptions about X as 
possible. One popular such general assumption about a time series is that it is 
a general linear process. 


Definition. The time series X is said to be a general linear process with 
coefficients 3 and errors «€ if 


X(t)=y+ D> Be(t-j), te Z, 


j=—o0o 


co 


where € ~ WN(a”) and the (’s are square summable, that is, )~° B? <BOOs 


j=—00 


Note that the square summability of the 6’s means that X (t) exists as a 
limit in mean square. The following theorem presents some properties of the 
general linear process that will be useful in Chapter 3. 
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Theorem 2.5.8 | PROPERTIES OF THE GENERAL LINEAR PROCESS 


If X is a general linear process, then 


a) R(v) =o? ga Bi Bj+Io}, ve Z. 


b) R is absolutely summable and thus 


2 
fe) = 078) Bye" | tere [0,1]. 


j=-0o 


Note that part (b) follows from the fact that a convolution (such as R(v)) 
of two sequences is absolutely summable if the individual sequences (in this case 
the sequence of (’s with itself) are absolutely summable (see Fuller (1976), p. 
28). It is also easy to see that ARMA processes are general linear processes. 


2.5.10. Nonlinear and Non-Gaussian Processes 


In this book we are concerned primarily with processes that can be ex- 
pressed in the linear form described in Section 2.5.9. A wide variety of authors 
are currently investigating the analysis of processes that are not linear and/or 
not Gaussian. A simple example of such a process is given in Problem C2.10. 
The interested reader can consult Section 11.5 of Priestley (1981) for a general 
introduction to nonlinear processes. 


2.6. Algorithms for Univariate Time Series 


In this section we discuss some of the algorithms that are used in univariate 
time series analysis. Thus the reader may want to give this section a cursory 
reading to get an overall view of the algorithms and how they are used in 
the TIMESLAB commands unless particularly interested in the details of the 
computations in TIMESLAB. 


We begin by considering methods for solving prediction normal equations. 
These algorithms yield a host of information useful in performing other tasks as 
well. While there is a vast literature on prediction algorithms, most attention 
has been paid by statisticians to the use of either Cholesky factorization or 
the Kalman Filter Algorithm. While both of these procedures are calculating 
the same quantities, we will study both as each makes clear what happens in 
prediction in a different way. 
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2.6.1. Prediction Using the Cholesky Decomposition 


We can use the modified Cholesky decomposition (MCD, see Section A.1.1) 
to obtain a series of results about calculating finite memory predictors. For 
the first, we follow Newton and Pagano (1982). 


Theorem 2.6.1 | PREDICTION VIA THE MCD 


Let X be a purely nondeterministic covariance stationary time series with 
autocovariance function R and let y) = 1,71,... be the coefficients of the 
MA(oo) representation of X. For K = 1,2,..., let 


Tx = Toepl(R(0),...,R(K —1)) 


be the covariance matrix of Xx = (X(1),... ,X(K))?, andl, = Lx DxLt be 
the MCD of fx. Define ex = (e(1),...,e(K))? as the solution to Lkex = Xx 
and note that the vectors e;,e2,... are also nested. Then the finite memory 


best linear unbiased predictors and prediction error variances are given by, for 
eae 


° h-1 
a) Xnk = Bs Lnthnta—Ke(n + h—k). 
b)id2 i) Ee D 
) Onhk = 2uk=0 n+h,n+h—k/*n+h—k,n+h—k- 
c) Further, 
im, Ln nj =e J 2 1 


lim Dan = 02.,. 
n—oo ae ool 


Part (c) gives a method for finding the y’s if we know the function R. If we 
compare parts (a) and (b) to the corresponding formulas for infinite memory 
prediction in part (d) of Theorem 2.4.1, we see how the elements of the MCD 
are playing the role for finite memory predictors that the y’s do for infinite 
memory prediction. This theorem also gives us one method of arriving at what 
is called the prediction error variance representation of the Gaussian likelihood 
of the parameters of time series. This likelihood is actually the joint pdf of a 
finite realization from a series. 
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Theorem 2.6.2 | GAUSSIAN LIKELIHOOD FUNCTION 


Under the conditions and notation of Theorem 2.6.1., we have 
i (i); pew 


XG) = Xa el 


a) e(j) = 


RO), Ged 


Ja aaa 


c) If X is Gaussian, then the joint pdf of X,, also called the likelihood 
function, is given by 


-1/2 
“e = 1 
fix)iai (2a) a4 (i Ds] exp (- DES a), 


that is, the pdf is only a function of X(1), R(0), and the one step ahead 
prediction errors e and error variances D;; for j = 2,...,n. 


Proof: To prove part (a), note that e(1) = L7’X, = X(1), while for j > 1, 
we have that X(j) is the jth row of L; times the vector e;; that is, 


y 
X(j) = 5 Lje(l) 
Heme} 


jzi 
= e(j)+ >) Lje(!) 
l=1 
= e(j) a7 Reavis 
since L;; = 1 and by part (a) of Theorem 2.6.1. For Hee (b), we have 


L,D,Lf = Dy, =T = R(0), while for j > 1, Dj; = 63_,, by part (b) 
of Theorem 2.6.1. For part (c), we have (see Seaon A.4. 9)? 


f(xe) = (2n)-"/?|P|-¥? exp —3x7 51x 


But 
n 
IP. |= nD.t2 |= (Lp | nlGel (Dal = PPPs, 
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since the determinant of a triangular matrix is just the product of its diagonal 
elements (which in the case of L,, are just ones). Further, fz! = L= 7 DetL*. 
and thus 

xT oe x LD, Lx =i D-1z 


’ 
where z = L71x = ey. 


Part (c) is important for finding maximum likelihood estimators for the 
parameters of ARMA processes. We will see that the Kalman Filter Algorithm 
is ideally suited for calculating the e’s and D’s. 


The Levinson Recursion 


In the previous two theorems we found finite memory linear predictors 
and prediction error variances without actually calculating the prediction co- 
efficients A,,. When h = 1 there is a well known algorithm due to Levinson 
(1947) for recursively calculating these coefficients for memory 1,2,.... This 
algorithm is based on the Toeplitz form of the covariance matrix of a covariance 
stationary time series (see Friedlander et al. (1979) for a general description 
of algorithms for Toeplitz and near-Toeplitz matrices), and as we shall see is 
very important in a number of contexts. In the following theorem we omit the 
second subscript on An, and &?, since h = 1 throughout. 


Theorem 2.6.3. | THE LEVINSON RECURSION 


Let X be a purely nondeterministic time series with autocovariance func- 
tion R and let 


r,, = Toepl(R(0),...,R(n — 1)) = L,D, L? 


be the MCD of the covariance matrix of X, = (X(1),...,X(n))?. Let 
A1,A2,... be the prediction coefficients of X41, X2,..., and let 


oF = Var(X (j + 1) — Xj), 


with 68 = R(0). Then 


a) Forn > 1, 
1 ~A;(1)  —A2(2) —An-1(n — 1) 
1 —A2(1) —An-1(n — 2) 
he 1 Tapes — 3) | 
Ayal) 


122 UNIVARIATE TIME SERIES CH. 2: 


that is, the (j,k)th element of U,, is 0 for j > k, 1 for j = k, and —\,_1(k—j) 
for j < k. Further, 


1h a! 
De Vie Did 
n n iaglsa) 5a noe ) 


b) The X’s and G?’s satisfy the following recursion for j > 1: 


Rj +1) — Dhar AV(A)RG +1 - ) 


a2 
oj 


Aj4i(g +1) = 


Aji(k) = AZ(K) — Aji + 1)AKG+1—k), k= 1,...,3 

oF 44 = aF(1 = N15 1))5 
with \4(1) = p(1), 62 = R(0), and g? = o3(1 — A?(1)). 
Proof: To show part (a) we need only show that '7! = U,V, U2 or equiva- 
lently that T,,U,V,U! = Ip, since then T, = U7? V71U;-1, and since U-? 
is unit lower triangular, V;+ is diagonal with positive diagonal elements, and 
the MCD is unique, we must have L, = Uz! and D, = Vz71. We show 


DU sus = I,, by induction. The claim is clearly true when n = 1. For 
k > 1, define r, = (R(1),...,R(k))?. Then 


I, Pyr; U, —P.r», 
Tye41Uk41 = 
z rZ7P, R(0)| {oF 1 
| T,U, Pyrr, pein 


rf PU, R(0) Sap TP? d, 


T,U; Ox 
Lr? P,.U, 6? 
by part (b) of Theorem 2.4.1 and the properties of the permutation matrix. 
Thus 


VU) <0; 


I, U;, a : 
a Fis 


T 
Pe 41U 41 Ve41 Un 4 = ‘ * 
rp P,U; CO; 


1 1 
—-=xIP, = 
rd hie Bey 

by the induction hypothesis and the fact that 


re P,U,V,U? =r PDT, =ri Tz’ P, =APPx, 
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since P,T';*P, = T,' and by part (b) of Theorem 2.4.1. To prove part (b) we 
now use part (a) and more algebra to show that 


—1 
Acer =T yyy regi 


1 
Ve = PA Fac + 1) _ APans)| 
k 


1 

aes + 1) — AF Prk) 
which is part (b). 
Partial Autocorrelations 


The coefficients A; (1), A2(2),... play an important role in time series anal- 
ysis. The next theorem shows that they are in fact partial correlation coeffi- 
cients. The proof of the theorem is a straightforward application of the results 
of Theorem 2.6.3. 


Theorem 2.6.4 | PARTIAL AUTOCORRELATIONS 


Let X be a purely nondeterministic time series and let Km be a realization 
of length m from X. Let Xm+4ijm and Xo|m denote the best linear predictors 
of X(m +1) and X(0), respectively, given X,,. Let 

Emtijm = X(m+ 1) — Xa les and €9im = X(0) — Notes 
Then 

a) Xoupijen = Ace Xa eeeX om = AL Xn. 

b) Var(€m+1jm) = Var(eojm) = 62,41. 

c) Am41(m +1) = Corr(€m41|m;€ojm); that is, Am+i(m + 1) is the corre- 
lation between what is left in X(m-+ 1) and X(0) after having removed their 


linear relationship with the X’s in between, that is, with X,. Thus A,,(m) is 
called the partial autocorrelation coefficient of lag m. 


Note that 


Xm4ijm = Am(1)X(m) +--+ + Am(m)X(1) 


Xoim = Am(m)X (m) + «+» + Am(1)X(1); 
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that is, the forward prediction coefficients are just the backward prediction 
coefficients in reverse order, while the forward and backward prediction error 
variances are the same. It is important to note that all of these results are 
based on the fact that the matrix I, in the prediction normal equations is 
Toeplitz. We will see sample analogs of these results in Section 3.4.4. 


Levinson’s Recursions and AR Processes 


In Section 2.5.4 we studied the autoregressive process of order p 
X (t) +a, X(t — 1) +---+apX(t — p) = c(t), 


where € ~ WN(o?) and the zeros of g(z) = Dj=0 a;2) are all greater than 


one in modulus. We showed that @ = (aj,..., ay)? satisfies the Yule-Walker 
equations 

T,a=-—r, 
Thus we have that a = —A, and we can use Levinson’s algorithm to calculate a. 
If we let aj; = (aj(1),...,a;(j))? = —A;, we can rewrite part (b) of Theorem 


2.6.3 as (omitting the hats on all of the quantities involved): 


Rj t= Se 1 05 (k)R(j + 1— k) 


2 
a) 


aj41(k) = aj(k) +ajg¢i(§+1)ajsG+1—k),. k=1,...,9 


ajqi(j+1) = 


oF41 = o3(1—94:(5 +1), 


with a;(1) = —p(1) and o§ = R(0), and note that a = ay and of =o”. 
It is also easy to see that for m > p, 


a 
am = ) 
Une 


and thus the structure of the MCD of T,, for an AR(p) process is very simple. 


THE MCD oF T, For an AR(p) 


Let 
X ~ AR(p,a, ays 


and for j > 1, let a@; and oF denote the solutions to 
Tja; = Tj 


ire Tp-1 
o; = R(0) —r; Tj °r;. 
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Let T,, = L,D,L? be the MCD of the covariance matrix of 


Xp =(X(1),.25;X(n))*. 


Then 
a) Form > p, 
a 
an = i 
OA, 
and 07, = 07, am(m) = —Am(m) = 0. 


b) For j > p+1, the jth row of L>! is given by 


i fh 
(One cea a: eee >}, 1,0,_-;); 


while the upper left-hand (p+ 1 x p+ 1) corner of L=? is the matrix 


1 
ai (1) 
a(2 ao(1 il 
re 2(2) 2( ) 
Qp-1(p—1) ap-i(p—2) ... Teh Peet 
Op Op-1 sieves a2 On tL 


Thus there are only p+(p—1)p/2 distinct elements in L;'. We will use this 
fact to simulate data from a Gaussian AR process and when doing regression 
analysis for a model having AR errors. 


Testing AR Processes for Stationarity 


Given coefficients a1,... ,@_, we would like a simple means for testing 
whether the zeros of g(z) = rae aj;z) are all outside the unit circle. One 
obvious test is to actually find the zeros (see the TIMESLAB command POLY- 
ROOTS) but there is another simpler test which is contained in the next theorem. 


Theorem 2.6.6 | THE ScHuR TEST For STATIONARITY 


Let a,...,a@p) be a set of constants, and let U and V be the (p x p) 
lower triangular Toeplitz matrices having first columns (1,@1,...,@p-1)? and 
(ap,...,a1)7, respectively. Then 
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a) The zeros of g(z) = DoySt are all greater than one in modulus if and 
only if the (p x p) matrix 


S = UU? —- vv" 
is positive definite. 
b) If S is positive definite, then 
iso's 
is the covariance matrix of an AR(p,a@,o7) process. 


c) T, is positive definite if and only if the partial autocorrelations of lags 
1 through p are all less than one in absolute value. 


Note that part (a) is due to Schur (see Pagano (1973)), and thus we will 
refer to the matrix S as the Schur matrix of a. The results of this theorem 
suggest a simple test for whether the zeros of g are all outside the unit circle. 
This check is incorporated in the ARPART command. 


The ARPART Command 


Given a set of coefficients a),...,a,, we can test the zeros of g and obtain 
the partial autocorrelations as a byproduct by essentially running Levinson’s 
algorithm backward. For 1 < k < j, we have 


Aj (F) = Aj4a(k) + Aj4i (9 + 1)Aj(G +1 — k) 
Aj(G +1 — kb) = Ajga(G +1 — k) + Aja (§ + 1)Aj(K), 
which gives 


A541(k) + Aj4a (9 + 1)Aj4i1(7 + 1- &) 


Aj(k) = 
i(®) 1 — AF41(5 + 1) | 


(ReSlleaen si fe 


The algorithm starts with A,(k) = —a,x and proceeds for j = p—1,...,1. If 
any of the partials are greater than one in absolute value, then we know that 
the zeros of g are not all outside the unit circle. 

In the command 


part=ARPART(alpha,p,ier) 
the output integer variable ier is zero if none of the partials are greater than 


one in absolute value, and is equal to j if 4;(j) is the first value (starting with 
j = p) greater than one in absolute value. 
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The PARTAR Command 


We can also easily derive an algorithm for finding the coefficients @ of an 
AR(p) process corresponding to a set of partial autocorrelations. Thus given 
A1(1),.-.,Ap(p), we can find recursively 


Aji (k) = Aj(k) + AjHA(G+DYAQG+1—-k),  k=1,...,5, 
for j =1,...,p—1, and then aj; = —A,(j), for j = 1,...,p. The command 
alpha=PARTAR(part,p) 


carries out this algorithm. Note that PARTAR does not check whether or not 
the absolute values of part are less than one. 


The ARCORR Command 


The results of the last few theorems also suggest a method for finding 
the autocorrelations of an AR(p, a, a”) process given its parameters @ and o2. 
First we have 


et 
But o2 = R(0) and so 
FS 
R(0) = =. 
[[@ - o7()) 
j=1 
Next we have that a; satisfies .;a; = —r;, the last row of which gives 


Rij) =—)ooaj()RG-1), 7 >1. 


i=1 


Then p(v) = R(v)/R(0). Thus the algorithm consists of (1) running Levin- 
son’s algorithm backward as in ARPART until R(0) is determined, (2) running 
Levinson’s algorithm forward for j = 1,...,p, at each order j finding R(j), and 
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finally (3) if the number M of desired correlations is greater than p, using the 
Yule-Walker equation 


P 
Rip+k)=—-lajR(Qp+k-j), k>1, 
j=l 
to find the remaining R’s. Again in the command 


rho=ARCORR(alpha,p,rvar,M,RO,ier) 


ier indicates whether the zeros of g are all greater than one in absolute value 
in the same way as in ARPART. Note that M is the number of autocorrelations 
that are desired. 


The CORRAR Command 
The command 
alpha=CORRAR(rho,RO,p,rvar) 


uses Levinson’s algorithm to find AR parameters alpha and rvar from corre- 
lations rho and variance RO. 


The ARDT Command 


To simulate a Gaussian AR(p) process, ARDT generates the initial values X, 
to have covariance matrix I’, and then uses the AR difference equation to obtain 
the rest of the observations. Thus ARDT does Levinson’s algorithm backward 
to obtain L, and Dy, and then solves the triangular system of equations (see 
Theorem 2.6.5) 

Death, Xp = 65; 


where ep ~ N,(0p,I,), that is, 
e(1)VDu, j=l 
X (4) = 4 (4) Vij — DA 5 -A(W)X(G- 4), G= 2-50 
e(j)Vo? — Yay a(k)X (7 —&), jeptt,...% 
This is done in the form 
x=ARDT(alpha,p,rvar,seed,n,ier,RO) 


which returns the stationarity check ier and the variance of the process RO 
as a byproduct. If the process is not stationary, then ARDT will abort and not 
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generate a realization. The ARDT command can also be used to generate values 
of any difference equation of the form 


X (t) +a,X(t — 1) +---+apX(t — p) = €(t) 


for any user-supplied X(1),...,X(p) and «(p+ 1),...,¢(n). The first p X’s 
and e(p+1),...,¢(n) are entered in the single array f in the command 


x=ARDT(alpha,p,n,f) 
and the initial values and generated values are returned in the array x. 


The ARFILT Command 


In Section A.5.3 we discuss transforming a regression model 
y=XBt+e, 


where Var(e) = 7?V, into one having uncorrelated errors by multiplying the 
observation vector y and design matrix X by D~/?L~-!, that is, by the inverse 
square root of V. The ARFILT command does such a transformation when V 
is the (n x n) covariance matrix of an AR(p, a, 07) process. In Section 3.7.1 we 
give a macro for solving a regression problem that has autoregressive errors. 


The MCD for ARMA Processes 


We have seen that the Cholesky factors for an AR covariance matrix take 
on a very special form. For the ARMA case, the lower triangular matrix L, 
can be further decomposed into two parts, one corresponding to the AR part 
of the model and one corresponding to the MA part. 

It would be nice if these two parts corresponded exactly to the triangular 
factors for an AR(p,a,o”) and an MA(q,f,o7), respectively, but this is not 
possible. There are two basic approaches to this problem; the first is due to 
Ansley (1979) and the second to Newton and Pagano (1982). The next theorem 
describes the second approach. It essentially decomposes L, into the product 
of the L for the AR part of the process and a matrix that is almost the L for 
the MA part of the process. 


Theorem 2.6.7 | ARMA PREDICTION VIA THE MCD 


Let X ~ ARMA(p,q,@, 8,07) and Z ~ AR(p,a,o?). Let 
T'x,n = Toepl(Rx(0),...,Rx(n—1)) =LxnDxnLX, 


and 
T'z.n = Toepl(Rz(0),...,Rz(n—1)) = LznDznLZn 
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be the MCD of the covariance matrices of realizations X,, and Z, of length n 
from X and Z, respectively. Define Y, and e, by 


Yue Ue Xe oar seen ky Xp, 
where 
Pyyn = Var(Yn) = L770 xnLz4, = LypDynLYn. 


Then 

a) The sequences Tx n, Tyn, zn, Lxn, Lyn, Lzn, Dxn, and Dyn 
of matrices and X,, Yn, and e, of vectors are all nested with respect to n, 
and thus for any n greater than or equal to both j and k, we can refer to the 
(j, #)th element of one of the matrices by dropping the subscript n and adding 
subscripts j and k. For example, Ly; refers to the (j,k)th element of Ly,, 


for any n greater than or equal to both j and k. Further, we can refer to the 
jth element of the vectors by X(j), Y(j), and e(j) for any n > j. 


b) Lxn = Lz pLyn and =Dxyn=Dypn. 


c) For j > p+4q, the only nonzero elements of the jth row of Ly,, are 
Ly; ;-~ fork =0,...,q. Further, 


din) Dy 752k = Be; R= UR 
joo 


: ee: 
lim Dy jj =o". 
j7oo 


d) The memory n, h step ahead predictor Xna, and prediction error vari- 
ance G2, are given by 


P 
Xnhk = Yar — > jkr 
j=1 


h-1 


ray I ae 2 
onh = » LX nthnth—-kDX.n+h—-k,nt+h—k; 
k=0 


where 


a Seat Ly nt+anth—ke(n +h — k), h = Dees ¢ 
Gi — 
0, h>q 
and Xnn-j = X(n+h—jJ) ifj >h. 
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e) Ifq =0, that is, if X ~ AR(p, a, 0”), 


j=1 
h-1 h-1 
x2 2 = 2 : 
Teh = 2 aes ey ere a = yu, ifn > p, 
k=0 k=0 


where the y’s are the coefficients of the MA(co ) representation of Z. 


f) If p = 0, that is, if X ~ MA(q,B, oA): 


KR Diss Lx nthnth—Ke(n + h — k), v= One g 
Xnh = 
0, h>q 

h-1 
A 2 
orn = NS ee) 

k=0 

and Lxnn—k > Be, fork =0,...,q, while Dxnn > o? asn— oo. 


2.6.2. ARMA Prediction and the Kalman Filter Algorithm 
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Under some circumstances, the best linear unbiased predictors can be 


calculated very simply. An example is the Gaussian AR(1) process 


X(t) + aX(t — 1) = €(t), 


since then X is a Markov process; that is, the conditional distribution of X (n+ 
1) given X(1),...,X(n) is the same as the conditional distribution of X (n+1) 
given just X(n). This is intuitively clear and easily verified by finding the two 


distributions involved (see Problem T2.16). 
Thus for a Gaussian AR(1) we have 


Xn = E[X(n + 1)|X(1)....,X(n)] 
= E[X(n + 1)|X(n)] 
= E[-aX(n) + e(n + 1)|X(n)] 
= —aX(n), 
since €(n + 1) is independent of X(n). Further, 


Xnh = —aXnh—-1 = ++ = (—a)*X(n). 
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Now if X satisfies a finite order difference equation, we will be able to construct 
a vector time series that is Markovian (see Problem T4.10) and use arguments 
similar to those above for an AR(1) to get predictors. The Kalman Filter Al- 
gorithm (see Section A.4.3) is ideal for doing prediction given a vector Markov 
process. 


The Markovian Representation of an ARMA Process 


There are several ways to determine a state and observational equation 
for an ARMA process. We describe the one given in Jones (1980). 


Theorem 2.6.8 | ARMA STATE AND OBSERVATION EQUATIONS 


Let X be an ARMA(p,q, a, 8,07) and let 
Xt,0 
Xt 

Xt = ° ) 
Xtr-1 
where r = max(p,q+1) and X¢p is the infinite memory, h step ahead predictor 
of X(t+h). Then 
a) The X; satisfy the state and observational equations: 


X, = AX_1 + we 
X (t) = he xX + Ut; 


where h is a vector of length r consisting of zeros except for its first element 
which is one, Var(v;) = 0, and w; = ge(t) with g = (yo, 71,---,%r-1), where 
the y’s are the coefficients in the MA(co) representation of X , while 


0 1 0 oa 0 0 
0 0 1 0 0 
A= De wt 
0 0 0 0 1 
Ap —Ap-j~ —Ar_2 ag 2. Meco 


and a, = 0 ifk > p. 


b) E(Xo) = 0,, while the (j,k)th element of Var(Xo) is given by 
min(j—1,k—1) 


Dj = R(|j — kl) — 0? ry WV+1j5-k] 
i=0 
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which is R(0) if j =k. 


Proof: From part (d) of Theorem 2.4.1 we have that the infinite memory, h 


step ahead predictor X;, of X(t +h) and its error mean square o7, are given 
by 


Xth = >> yee(t +h — k) 
k=h 


h-1 
ape 2 
Cth = 9 5 th 

k=0 


where the 7,’s are the coefficients in the MA(oo) representation of X. Thus the 
first element of X; is just X(t) which verifies that the observational equation 
above is valid. Note how simple this equation is. The sequence of matrices H; 
in the general formulation is the simple constant row vector h7 , d is one, and 
the observational error covariance matrix V; is just a scalar zero. 

We have for j7 = 0,1,...,r—1 that 


Xe,j = Vg4E(t) + yj41€(t — 1) + y42e(t — 2) +--- 


Xe-1,541 = Wy41€(t — 1) + y42€(t — 2) +---, 


that is, X;; = Xt-1,541 + 7j¢(t). This gives a recursion for the elements of X; 
in terms of those in X;_; except for the last element of X, which is in terms 
of Xt_1,r which is not an element of X;_;. But 


Xt—1. = E(X(t—1+r)le(t — 1), €(t — 2),...) 


= —)) ajE(X(t — 14 (r— j))|e(t — 1), €(t — 2),.. 5) 
j= 


+>) PE(e(t—1+1r—k)|e(t — 1),€(t —2),...). 


k=0 


Since r = max(p,qg +1), we have r > k in the second sum and thus each term 
in the sum is zero. Finally, the e’s up to time t — 1 are just a linear function of 
the X’s up to time t — 1, so we have 


, Cas ey = E(X(t Lae (r at I))IX(t sa 1), X(t Di 2), 3 ) 
= E(X(¢— 1+ (r—§))|e(t — 1), e(t — 2),....), 


and since r > p, we have t—1+(r— j) > t —1 which means each term in the 
first sum is an element of X;_;. Thus we have verified that the state equation 
is true. 
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It remains to verify that the starting values in part (b) are valid. The mean 
of X; is 0, since its elements are infinite memory predictors. The formula for 
the covariance matrix follows immediately from part (d,ii) of Theorem 2.4.1. 


Implications: From the Kalman Filter Theorem (see Theorem A.4.4) we have 
that the first element of X;4;~-1j:-1 and the (1,1) element of Dy4,~1)¢-1 are 
the memory t¢ — 1, h step ahead predictor and prediction error variance of 
X(t+h—1) given X(1),...,X(t— 1). To evaluate the Gaussian likelihood of 
an ARMA process we need only these terms for h = 1 (see Theorem 2.6.2). 
Note that the transition matrix A is the same for all ¢ and has mostly zeros as 
elements. 


The ARMAPRED Command 


Given data X(1),...,X(n) and the parameters of an ARMA process, 
ARMAPRED calculates memory t, h step ahead predictors and (optionally) pre- 
diction standard errors fort =tys,...,t; and h=hy,...,h;. Note that ty > 1, 
t; <n, and h; > 1. 


2.6.3. The Cramer-Wold Factorization 


An important problem in time series analysis is to find moving aver- 
age parameters B and o? corresponding to a given set of autocovariances 
R(0), R(1),...,R(q). We will devote an unusual amount of attention to this 
problem because the algorithm usually used to compute it, and others that are 
very similar to it, are heavily used in a variety of ways in modern time series 
analysis. 


The covariance generating function of X is 


q 


T(z) = D7 R(v)2” = 07(z)h(z)A(z~), 


v=-q 


where h(z) = )~7_, Gxz*. Thus finding B and o? is in fact finding a fac- 
torization of [. This factorization is usually referred to as the Cramer-Wold 


factorization. There are two well known facts about this problem (see Wilson 
(1979)): 


1. There is no real solution unless the spectral density f(w) = T'(e?™*”) > 0 
for all w € (0, 1). 


2. If f(w) > 0, then there are 2? sets of MA coefficients that all give the same 
covariances, but only one of these sets will be such that h(z) has all of its 
zeros outside the unit circle. This factor is called the Hurwitz factor of T. 
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Bauer’s Algorithm 


One solution to the problem of finding the Hurwitz factor is due to Bauer 
(1955). We have from Theorem 2.6.1 that if we let 


I'y = Toepl(R(0),..., R(q),0,...,0) = Ly Dy LZ, 


then 
lim DL; ,-; = B; 
ke kk—-j = 8; 


lim Dx = C7 

k- 00 
In Example 2.8, we give a macro illustrating this algorithm. Unfortunately, 
Bauer’s algorithm can converge very slowly (see Pagano (1976)); that is, the 
elements of Ly can take a long time to converge to the (’s. Further, there is 


no easily applied check for determining if the factorization is feasible, that is, 
if convergence can be reached at all. 


Wilson’s Algorithm 


The algorithm derived by Wilson (1969,1979) converges rapidly and does 
have an easily applied check for the feasibility of the factorization. Wilson’s 
algorithm is a straightforward application of the Newton-Raphson method for 
solving for the parameters a? and (j,..., Bq in 


q—|v| 


ei > BeBetlo| — R(v) = 0, Rah are) 
k=0 


or, if we let 7; = 08;, 7 =0,...,q, for solving for Tyree pte ll 
q—l»| 
Coke yi TkTk+|v] — R(v) = 0, Vv Use- 0" 
k=0 
Wilson suggests using 7,0 = \/R(0) and 7, = Hi) /T4, j = 1;.-., 0.88 
the initial approximation To to T = (7,..., T )?. Then the Newton-Raphson 


iteration step is given by 
ne Peet Gee: 
Ti41 = Tj i Ci, 


where 
q—|v| 
Ch = Ser Ti,kTi,k+|v| — R(v) 


k=0 
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is the vth element of c; and G; is the (q+ 1x q+ 1) matrix (evaluated at 
T;) whose (j,k)th element is 0C;/07,. It is easy to see that G; = T,; + To; 
where 


Ts Op TSN) Gi-fets emcee Taig 
T1 71,2 +--+ Ti,g 
Tig= 
Tig 
and 
Ue) up Scidioe con Tig 
Ti,0 +++ +++ Ti,q—1 
Toi= 
7,0 


Thus each iteration consists of finding 7;41 = T;—d; where dj = Gj 16; is found 
by solving the (¢+1xq+1) system of equations G;d; = c;. In general, solving 
such a set of equations requires on the order of g® operations (multiplications 
and additions). We next show how this system of equations can be solved in 
order q? operations. 


Derivation of the Updating Step in Wilson’s Algorithm 


The key to Wilson’s new (1979) updating step in the Newton-Raphson 
algorithm is to note that there is a Levinson-type recursion available to obtain 
Ti41 from 7;. 

We first derive a polynomial equation that is equivalent to the system 
of equations that we want to solve. We have from Tj; = T; — G> le; and 
G; = T1; + To; that 


Tr iTiga + T2iTig. = T1i7i + Toit; — ci. 


But it is easy to see by the form of T2; that c; = T2;7; — R, where R = 
(R(0),...,R(q))7, which gives 


TisTi4a + Te iTig¢. = Titi t R= RR; +R, 
where R; = (R;(0),..., Ri(q))? and 


q—|v| 
R,(v) = Sa) TikTi,k+|ul> v=0,...,9. 
k=0 
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Now we note that two (q¢+1)-dimensional vectors a and b are equal if and only 
if : 4 
a2) = ay} Oia ao bjx\z* = b(z), zEC. 


k=-q k=-q 


If we let a= T1;7;41, then a(z) = c(z)d(z~1), where 


q q 
Elz Wie and d(z) = iiaz 
k=0 k=0 


Thus we have 
e(z)d(2-) + e(2-1)d(z) = e(2), 


where 
q 


e(z) = > (Ri(k) + R(R)) 2. 
k=-q 
We know the coefficients of c(z) and e(z) and would like to find those of d(z). 
If we let f(z) = c(z)/7%,0 and r(z) = e(z)/7;,0, we can write 


( 
f(z)d(z) + f(2-*)d(z) = r(z), 


where the coefficients f1,..., fg of f(z) and those of r(z) are known, the coef- 
ficient of z° in f(z) is one, and we seek the coefficients of d(z), that is, r;4}. 
This is the polynomial equation that we referred to above. 

Define the coefficients f,; for 1 <j < k < q, by performing Levinson’s 
algorithm backward starting from fy; = fj = j/T,0, J =1,...,q; that is, 


_ Setigy — fetietifesiegi—j a9 
i his scat ie Sng peer POA GRE A= Lebo} q, 
Fa Feti,k41 


for k = q—1,...,1. Recall that this means that 
F+ig = fej + Setijetiteegi—j- 


Thus if we define the polynomial f,(z) = ie fiz, we have 


fi(z) = f(z) + frgajepi1z° tf, (277). 
Note that f(z) = f,(z) and thus we can write (letting k+1= q) 


r(z) = {fo-1(2) + faq2*fa-1(2-)} d(z-1) 
+ {fa-1(2~") + faq2~* fa-1(z)} d(z) 
= f-1(z) {d(2~*) + fa,q274d(z)} 

+ fg—1(2~*) {d(z) + faqz4d(z~*)} 


— fq—1(z)hg(z7*) cts fgas(z7*)hg(2)s 
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where hq(z) = d(z) + fg,qz7d(z~*) is a polynomial of degree q. 

The highest power of z in fy_1(z)hg(z7+) is g—1, while in f,_1(z~*)hg(z) 
it is g which has coefficient equal to that of z? in hg(z), which we will call 7g, 
which has value 7)41,¢ + fg,q7i+1,0- But 7g is also equal to the coefficient of z? 
in r(z), that is, yg = (Ri(q) + R(q)) /7,0- 

Now define the polynomial dj_i(z) of degree g — 1 having coefficients 
eet ty cee at: erie | and satisfying 


dg—1(z) = hg(z) — ¥q24 = d(z) + faqztd(z~*) — Yq24. 


Thus 
dq-1,j = dq,j 7 fa,a4q,9-3; co PH Blatt bs 


where the coefficients of d(z) (which are what we want) are denoted by d,,;. 
Replacing j by q — j in this equation gives 


dg—1,q-j — dq,q-j a3 Ffa,944,3; j= Oo ig —) |, 


and if we solve this for dg ; and substitute the result into the equation for dg_1; 
we get 
do = dg—1,5 — faq,q4q-1,9-5 
9535, Srp ’ 
1— fig 


Except for j = q, this is a recursion for the d’s. We also have that 7, is the 
coefficient of hg(z) = d(z) + fq,qz%d(z~1), that is, yg = dg.g + fg,q¢q,0, Which 
means that if we define dg_1,q = Yq, then the above equation for d,,; holds for 
j=q as well. 

The next step in deriving the algorithm is to note that substituting hg(z) = 
dg-1(z) + 742% into the equation for r(z) gives 


fosle)dgae) tg fg-1(27*)dg_i(2) = Pa(%); 


hee Uae ane 


where rg_1 = r(z)—Yqz~9 fg—1(2) —¥q2% fg-1(27*) has powers of z from —(q—1) 
to g—1. This equation is identical in form to the one we started with except 
that the degrees have been reduced from q to q — 1. The function rg_;(z) has 
coefficients 


Tq-1j =%q-1,-5, J=0,...,¢-1, 
where rg_1,9 = q,0 and 
Tq-1,5 = 74,5 — Yafq-1,9-3) g=1)...,9—1, 


if we define rqg9,...,1q,q to be the coefficients of r(z). 
We can now repeat this process until we get 


fo(z)do(z~*) + fo(z~*)do(z) = ro(z), 


that is, doo = r0,0/2. 
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Summary of the Updating Step 


Thus we have the following algorithm for obtaining 7;4; from 7;. First 
find 


ia : 
fasitsagin Jala 
Ri) + he! hat et 
eS hte initia: 
Them tor k= @,-.:, 1° 
Yk = Tk,k 


ae Oe Treg — Seek Fkeyk—j 
Sem = yi} ) 
|e 


TE=1 — Tep RRO Rs FST kt. 


Finally, doo = rq,0/2, and fork = 0,...,q—1: 
dk k4+1 = Vet 


dk — Se4i,k419k,k41-5 
eed ESET URIS EELS i) 


dk41j = i-f 
k+1,k+1 


andutsy 97 =—.dg 5, 9 = 0,ers 19: 

One final point is that Wilson shows that the Cramer-Wold factorization 
is not feasible either if T;,0 18 negative for any iteration or if one of the tk,k’S 
(which are analogous to partial autocorrelations) is not in (—1, 1). 


The CORRMA Command 


The command 
beta=CORRMA(rho,q,RO,maxit ,del,ier,rvar) 


performs the algorithm that we have just described. The input is the order 
q, the array rho which contains p(1),...,p(q), the variance RO, the integer 
maxit which is the maximum number of Newton-Raphson steps that the user 
will allow, and the real scalar del which is a convergence criterion. Iterations 
continue until either | 
nie et. 

_max Mita) — Til < del, 

j3=0,...,¢ ere 
in which case the output integer variable ier is set equal to 0 and beta and 
rvar are formed, or the maximum number of iterations is reached, in which 
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case ier is set equal to 1, and beta and rvar are not formed. Finally, if at 
any iteration, the factorization is deemed unfeasible, ier is set equal to 2, and 
CORRMA aborts. For most cases, a value of 20 or 30 for maxit with del= 107° 
should give satisfactory results. 


2.6.4. Factoring an ARMA Covariance Generating Function 


Suppose we are given the first M+1 autocovariances R(0), R(1),..., R(M) 
of an ARMA(p, g, @, 8, 7”) process, and we would like to determine the coeffi- 
cients and noise variance for the process. Note that this is problem is essentially 
finding the factors g(z) and h(z) in the ARMA covariance generating function 
(see Theorem 2.5.6) 


=i 
r(-) = Mea) 
g(z)g(z-*) 
We know from Theorem 2.5.6 that we can obtain a;,...,@p from the so-called 


high order Yule-Walker equations 


P 
>| aj R(v — j) = 0, v=qtl,...,qg+p?. 
j=0 


This results in the (p x p) system of equations 


R(q) eG Pp) roa R(q +1) 


Rp 1) (g) Ap R(q +p) 


Thus we need that M > p+ q in order to solve this system. 
If we write 


P 
Y(t) =} 0 aj;X(t- 3) = YA (t—k 
j=0 


then we have from Y(t) = )~7_, Gee(t — k) that Y is an MA(q, B, o”) process, 
while from Y(t) = ees a;X(t —j) and the Filter Theorem, we have 


Ry(v) =E(¥()¥(t +») 


= DD a AnE (X(t — 5) X(t - f)) 


P > P 
= >So aja, R(j +v—h), u> 0: 
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Thus we can calculate Ry (0), Ry(1),..., Ry(q) from this and then use Wilson’s 
algorithm to find the 6’s and o?. 


The CORRARMA Command 
The command 
rvar=CORRARMA(rho,RO,p,q,maxit,del,ier, alpha, beta) 


performs the method described above. The input arguments maxit and del are 
the same as described in the CORRMA command, while the output integer ier is 
0 if CORRARMA ends successfully, 1 if Wilson’s algorithm does not converge, 2 if 
the factorization in Wilson’s algorithm is judged infeasible, and 3 if the matrix 
in the high order Yule-Walker equations is judged to be singular. 


The ARMACORR Command 


Suppose X ~ ARMA(p,q,@, 8,07) and we would like to find the corre- 
sponding autocovariances R(0), R(1),...,R(M). We describe the algorithm 
given by Wilson (1979). 

If M > q, we can use the fact (see Theorem 2.5.6) that 


P 
Riv) =—)oajsR(v-j),  v >4q, 
j=l 


to find R(q + 1),...,R(M) if we first find R(q —p),...,R(q). In fact we will 
find R(0),...,R(r) for r = max(p,q) as this will guarantee that we will have 
all of the R’s that we need in order to use the difference equation (recall that 
R(v) = R(-v)). 

The covariance generating function I'(z) of X is given by 


2h(z)h(z~") 


T(z) = 0c? YD) 
= G(2)9(9) 
where g(z) = ae ajz) and h(z) = )~7_, Bez*. If we can find a polynomial 


7 of degree r having coefficients yo,..., 7, such that 


5(z) = h(z)h(z~") = g(z)y(27*) + 9(27")7(2), 


then 


1 Ve te) 

—T(z)= + —, 

att) = oat) + tz) 

and we can determine R(0), R(1),..., R(r) by equating coefficients of like pow- 
ers of z on both sides of this equation. Since the zeros of g are all outside the 
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unit circle, then 1/g(z) and thus 0(z) = y(z)/g(z) has only nonnegative powers 
of z. Similarly, y(z~1)/g(z~1) has only nonpositive powers of z. Thus we have 


R(v) = 06,  v>0, 


where 6, is the coefficient of z” in y(z)/g(z), while R(0) = 2070 , since 0p is the 
coefficient of z° in both y(z~1)/g(z~1) and y(z)/g(z). Now 0(z) = 7(z)/g(z) 
means that g(z)0(z) = 7(z); that is, 


o) r 
) aj0,25t* = eae 
1=0 


P 
j=0 k=0 


The coefficient of z* on the left hand side of this equation is age”, ) a55—5, 
while on the right hand side it is y,. This means that 0) = yo and 


min(s,p) 
0; =7s — ys aj9s_;, Fm a Se 
j=l 


Thus it remains to find yo,...,7, such that 


5(z) = A(z)h(z~*) = g(z)y(2*) + 9(27*)71(2)- 


But this equation is identical to the equation 
r(z) = f(z)d(z~*) + f(z") d(z) 


in Section 2.6.3, except that the degrees of the polynomials f and d there were 
q, while in the present case, g is of degree p while y is of degree r. Thus if 
r = p, that is, p > q, then the same algorithm as in Section 2.6.3 can be used 
with p replacing q and the coefficients of 6(z) and g(z) replacing those of r(z) 
and f(z). If p < q, we can still use the algorithm by using f,,; = 0 if k > p, 
and dk41,j = dy, j for k > p. 

The command 


rho=ARMACORR (alpha, beta,p,q,rvar,M,RO,ier) 
produces the variance R(0) and autocorrelations p(1),...,(M) in the scalar 


RO and the array rho. The output integer ier is 0 if ARMACORR finishes normally 
and is 1 if the process is judged to be nonstationary or noninvertible. 
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2.7. Examples and Problems 


Example 2.1 EIGENVALUES OF TOEPLITZ MATRICES 


There are situations in time series analysis where a knowledge of the eigen- 
values of an (n x n) covariance matrix is useful (see part (g) of Theorem A.4.2). 
In this example we illustrate the fact that these eigenvalues are related to the 
values of the spectral density at the n frequencies (j—1)/n for 3 =—T1,..., n. 

Let R and f be the autocovariance and spectral density functions of a 
covariance stationary time series and let 


Anj1 > An,2 > aa Ann 


be the eigenvalues of T,, = Toepl(R(0), R(1),..., R(n — 1)) arranged in nonin- 
creasing order. Suppose that m is the largest number such that f(w) > m for 
all w and M is the smallest number such that f (w) < M for allw. Suppose that 
M < o and let h be any continuous function defined on the interval [m, M] 
Let 


FQ) 2 fa) 2°" > Fin) 


be the values of f (arranged in nonincreasing order) at the frequencies (j—1)/n 
for j = 1,...,n. Then Grenander and Szegé (1958, p. 65) show that 


n 


and 


Thus if A is the identity function, we have that the eigenvalues and the sorted 
values of the spectral density are essentially the same for large n. Further, the 
integral of any continuous function of the spectral density of f (for example 
h(x) = x? or A(z) = logz) is also related to the eigenvalues of T,. 

The following macro will create and list the eigenvalues and sorted values 
of the spectral density for a user-specified AR(p) process. 


;; EIG.MAC: macro to illustrate the fact that the eigenvalues 
53 of a covariance matrix are related to the spectral 
density of the related process. 


shy INPUT: p, alpha (order and coefficients of the AR process to 
3 use for illustration) 


Noah OWN FF 
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8 35 n (size of covariance matrix to use. n must be .le.60) 
O35 

10 PAUSE 

11 ;start 

12 IF(n.1le.60,s1) 

1S 535 

14 ;; n must be less than or equal to 60 

153 5 

16 GOTO(end) 

is 

18 ;s1 

19 

20 rho=ARCORR(alpha,p,i,n,r0,ier) 

21 r=rho*r0 

22 gam=TOEPL(r,r0O,n) 

23 55 

24 cas Finding the eigenvalues takes some time.... 
25 


26 evals=EIG(gam,n,100,ier) 
27 f=ARSP(alpha,p,i,n) 


28 f1=f ;f1 is just here so we can use DOUBLE 
29 DOUBLE(f ,f1,n) ;form f at all n frequencies 

30 f=SORT(f,n) 

31 f£=REVERSE(f ,n) ;now f has spectra in descending order 


32 PROMPTOFF 

33 LIST(evals) 

34 LABEL(f)=’Values of f in descending order’ 
35 LIST(£) 

36 PROMPTON 

YG 

38 ;end 

39 ; 


To illustrate the use of this macro, Table 2.2 gives the result of using it for 
the AR(2) process having coefficients 0.3 and 0.9 (see Figure 2.2 for a plot of 
the log of the spectral density of this process). We have found the eigenvalues 
of the (40x40) covariance matrix of the process. Note the close agreement 
between the eigenvalues and the spectral density except for the eight largest 
values. 


Example 2.2 | THE SPECTRAL DENSITY FUNCTION 


We saw in Section 2.2 that the spectral density function of a time series can 
be thought of as the average over many long realizations of the periodogram. 
The PERAVE.MAC macro can be used to generate a series of realizations from a 
specified autoregressive process and keep a running average of the periodograms 
found for each one. There are four plots on the screen while the macro is 
running: (1) the true spectral density of the process, (2) the current data 
set, (3) the periodogram of the current data set, and (4) the average of the 
periodograms that have been produced. The point of the macro is that the 
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Table 2.2. An Example of the Relationship Between the Eigenvalues of a 


Toeplitz Matrix and the Corresponding Spectral Density 


Eigenvalues 

1 | 60.749760 60 .589540 21 .689460 21.507380 
6 | 8.837506 4.657563 4.537861 2.829798 
11 | 1.912571 1.811326 1.392952 1.298594 
16 | - 986147 - 856760 - 787423 - 703464 
21, il 586699 .573033 -512443 - 497300 
26 | -442225 -421246 - 417380 -401766 
31 | - 369590 - 328809 . 296827 . 271782 
36 | . 237020 . 225453 - 216951 - 211127 
Values of f in descending order 

1 | 102.427900 102.427900 10.930160 10.930160 
6 | 10.000000 3.082259 3.082259 2.729008 
teh | 1.484353 1.484353 1.256205 1.256205 
16 | - 914162 - 735545 - 735545 -651911 
21'5)| - 514863 - 514863 - 496563 -496563 
26 | - 440136 - 402263 - 402263 - 390625 
31 | - 369535 . 295989 . 295989 . 251651 
36 | . 225203 + 225203 - 211065 - 211065 


N 


10. 
- 729008 
-914162 
-651911 
-440136 
- 369535 
- 251651 
- 206612 


- 979041 
. 721278 
.071119 
-658951 
- 467412 
- 393535 
. 252204 
- 207729 


000000 


average of the periodograms gets closer and closer to the true spectral density 
as the number of realizations increases. Note that all of the spectral plots are 


on a log scale. 


si PERAVE: macro to illustrate the unbiasedness of the 
sue periodogram. For a specified AR(p) process, 
33 the macro generates nsamps samples of size n. 


a5 On the screen at the same time, the true AR 
a spectral density is shown together with the 


Ss average of all the periodograms so far. 


Pe 
rFoOooOmMANI Aah WN 


3; INPUT: n (sample size), p, alpha, nsamps 


DD as p, alpha (order and coefficients of AR process) 
13) ¥5\< nsamps (number of samples) 

i seed (random number generator seed) 

15 ;; 

16 PAUSE 

17 ;start 

18 PLOTON ;switch to graphics mode 

19 BATCHON ;no pause between graphs 

20 ns=1 ;initialize sample counter 

21 x=ARDT(alpha,p,1,seed,n,ier,r0true) ;find variance of process 
22 £=ARSP(alpha,p,1,n) ;find true f 

23 LABEL(f)=’True AR(#p#) Spectral Density’ 

24 PLOTSIZE(1) ;true spectra goes in upper left corner 
25 PLOTSP(f,n,rOtrue) ;plot it 


58 sample, the periodogram of the sample, and the 


146 


26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 


UNIVARIATE TIME SERIES CH. 
q={n/2}+1 
fave=LIWE(q,0,0) ;initialize average periodogram 
;startloop 
ERASE(2) j;erase old data 
PLOTSIZE(2) ;data go in lower left 
x=ARDT(alpha,p,1,0,n,ier,r0) 


LABEL(x)=?Sample Humber #ns#’ 


PLOT (x,n) ;plot it 
ERASE(3) jerase old periodogram 
PLOTSIZE(3) ;periodogram goes in upper right 


rho=CORR(x,n,0,n,1,r0,fh) 
LABEL (fh)=’Periodogram for Sample #ns#’ 


PLOTSP(fh,n,r0) jplot it 
fave={{ns-1}*fave/ns}+{fh/ns} ;find average 
ERASE(4) ;erase old average 
PLOTSIZE(4) ;average goes in lower right 
LABEL (fave)=’Average of #ns# Samples’ 

PLOTSP(fave ,n,rOtrue) 

if (ns .eq.nsamps, endloop) 3all finished? 
ns=nst1 ;no 
GOTO(startloop) 

; 

;endloop 

BATCHOFF syes 

GRMEWU(0, .5,-6,6) ;let user FIND, PRINT, etc 


oad 
FowWw MAN onh WN EH 


i 
CN ADAaA hk wON 


The macro DIRICH.MAC given below was used to generate Figure 2.3. 


re DIRICH.MAC: macro to calculate the Dirichlet kernel for 
ae user specified M for Q frequencies between 

A frequencies fi and f2 (inclusive). The 

ay kernel is then plotted with the vertical 

ats scale being between ymin and ymax. 

os INPUT: M, Q (number of frequencies between f1 and f2) 

= £1, £2, ymin, ymax (these can be entered as zeros) 
PAUSE 

;start 

freq=LIVE(Q,f1,£2,1) ;freq contains frequencies 
ff=pi+freq 

ff=REPLACE(ff ,Q,4,0,0,.0001) ;make sure frequency zero not included 
den=SIW (ff ,Q) ;denominator 

om=2 .*M+1 


SEC 


19 
20 
21 
22 
23 
24 


CWONA AHR WN eH 


w NN NON NY NHN NNN BH BP BB BH ep pe 


2.7 


ff=om*ff 
num=SIN (ff ,Q) 
dir=num/den 


LABEL(dir)=’Dirichlet Kernel For M=#M#? 


LABEL (freq) =? 


EXAMPLES AND PROBLEMS 


;humerator 


PLOT (freq,dir,Q,f1,£2, ymin, ymax) 


Example 2.4 | THE BANDPASS FILTER EXAMPLE 


The macros BAND.MAC and PARZ.MAC were used to form the graphs in 
Figure 2.4. 


3 BAND.MAC: macro to approximate the ideal transfer function 
of a bandpass filter centered at frequency .25 
and bandwidth .1. 


oe INPUT: m (number of terms in the approximation) 


PAUSE 
;start 


; calculate coefficients: 


u=.25 

v=.1 

bO=4.. *v 
ok=LIWE(m,0,1) 
argc=2*pi*ok*u 
args=2*pitok+*y 
c=COS (argc ,m) 
s=SIW (args ,m) 
b=2*c*s/{pi*ok} 
b=<b0 , b> 


MACRO(parz, start) 


bp=b+z 
nmm=200-m-1 


zero=LINE(nmm,0,0) 


b=<b, zero> 
bp=<bp, zero> 


, 


; Do unweighted transfer function and plot: 


, 
FFT(b,200,101,1,zr,zi) 
FFT(b,200,101 ,-1,zr1,zii) 


zr=zrtzri-boO 


x=LIWE(101,-.005, .005) 
xi1=<.15,.15,.35, .35> 


h1=<0,1,1,0> 
x2=<0,.5> 
h2=<0.0,0.0> 


3 Ok as £055. 5m 


; Sum for indices 0 though m 

; Sum for indices -m through 0 

; Add together and subtract 0 index term 
; Now get ready for plot: 


; b contains the m+i coefficients 

; The PARZ macro gets the m+i Parzen weights 
; bp has weighted coefficients 

; We’1l evaluate the transfer functions 

; at 200 points in 0 to 1 and plot 


; at the 101 points in 0 to .5. 
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40 xx=<x1,x2,x> 

41 yy=<hi,h2,zr> 

42 type=<2,2,2> 

43 n=<4,2,101> 

44 LABEL(xx)=’ ? 

45 LABEL(yy)=’No weights, M=#mi#’ 

46 PLOTK(xx,yy,n,3,type,0,.5,-.2,1.20) ;plot the unweighted approximation 
47 5 

48 ; Weighted Version: 

49 ; 

50 FFT(bp,200,101,1,zr,zi) 

51 FFT(bp,200,101,-1,zr1,zi1) 

52 zr=zrtzri-bO 

53 yy=<hi,h2,zr> 

54 LABEL(xx)=’ ” 

55 LABEL(yy)=’Parzen weights, M=#m#’ 

56 PLOTK(xx,yy,n,3,type,0,.5,-.20,1.20) ;plot the weighted approximation 


LS 

Pe oo PARZ.MAC: macro to calculate the Parzen weight function for 
i nS index 0,1,...,m and return it in z[i],...,z[mti]. 
Ass 

fg INPUT: m (must be even) 

6 33 

7 PAUSE 

8 ;start 

9 z=LINE(m,0,1) 

10 z=z/m 

11 mb2=m/2 

12 mb2pi=mb2+1 

13 z1=EXTRACT(z,1,mb2) ;z1 will get first half, z2 gets second half 


fy 
nS 


Z2=EXTRACT(z,mb2p1 ,m) 

z1=1-6*z1*{z-z"2} 

Z2=2*{1-z2}73 

z=<1,z1,z2> ; z(O)=1 

CLEAN (z1,z2,mb2,mb2p1) ;get rid of stuff we don’t need 


The RW.MAC was used (with n=200) to produce the random walk series in 
Figure 2.5. 


a 
onNnna 


;; RW.MAC: macro to plot 5 random walks of length n on the 
5 screen at once. 


33 INPUT: n (length of each random walk) 
35 seed (random number generator seed) 
si dist (which distribution to use in WH) 


Nonphbh wnrd 
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Example 2.6 GENERATING STABLE COEFFICIENTS 


O©mMANA AR WwW He 


eRe 
NF Oo 


PAUSE 

;start 

PROMPTOFF 

y=wn(seed,10) ;warm up random number generator 
sample#=1 


331 


LIST (sample#) 
yi=WH(0,n,dist) 
y1=CUM(yi,n,1) ;get next random walk 


y=<y1,y> ;append this one to the others 
IF (sample#.eq.5,plot) shave all 57 
sample#=sample#+1 ;no 

GOTO(s1) 

;plot ;yes 

x=LIWE(n,0,1) 


x=<x,xX,x,xX,XxX> 

type=<2,2,2,2,2> 

LABEL(y)=’5 Random Walks (n=#n#, dist=#dist#)? 
LABEL (x)=? 7 

PLOTK(x,y,n,5,type) 

PROMPTON 
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We are often interested in obtaining sets of coefficients whose characteristic 
polynomial has all of its roots outside the unit circle. The macro RANDCOEF .MAC 
given in this example will randomly generate such a set of coefficients. First 
the WN command is used to find a set of numbers between 0 and 1. These 
are then converted to numbers between —1 and 1, which we then treat as 
partial autocorrelations. From these partials we can find the corresponding 
autoregressive coefficients. These coefficients can be thought of as either AR, 
MA, or one part of a set of ARMA coefficients. 


Ae RANDCOEF.MAC: macro to generate a set of partial 


3 autocorrelations that are uniformly 


33 distributed in the interval (-1,1) 


oe and then find the corresponding AR 


aie coefficients. This can be used to 


a generate stable sets of AR, MA, or ARMA 


a coefficients. 


35 INPUT: seed (seed for random number generator) 
18 p (number of coefficients to generate) 
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3; OUTPUT: part (partials), coeffs (coefficients) 
a> 

PAUSE 

;start 

part=WN (seed,p,2) ;U(0,1) 

part=2*{part-.5} 3U(-1,1) 
coeffs=PARTAR(part,p) ;find coefficients 
LABEL(part)=’Partial Autocorrelations’ 

LABEL (coeffs)=’Coefficients’ 


Example 2.7 | PATTERNS IN ARMA PROCESSES 


sity 


We have seen that the correlogram, partial correlogram, and spectral den- 
functions of AR, MA, and ARMA processes have certain distinguishing 
characteristics. In this example we give a macro that can be used to study 
these characteristics. The macro is called IDARMA and can be used for AR, 
MA, or ARMA processes by specifying appropriate values for the orders of an 
ARMA process (p=0 for an MA process and q= 0 for an AR process). In each 
case, a series of coefficients are randomly generated, and for each set the cor- 
relogram, partial correlogram, spectral density, and one realization of length 
200 are displayed. The macro has an entry point labeled ;known that can be 
used if the user wants to generate the above quantities for a specified model 


and coefficients. 


omMNIaanrh wn 
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“oF IDARMA.MAC: Macro to illustrate the patterns in various diagnostics 
As for an ARMA(p,q) process. Successive coefficients are 
4 randomly generated and the properties of the resulting 
33 process displayed. 


ees HOTE: This macro will execute until it is broken by the user. 


Bi INPUT: p (O for MA processes), q (O for AR processes) 
rd seed (seed for random number generator) 

PAUSE 

jstart 

CLS 

part=WH (seed ,10) ;warm up generator 

; Randomly generate coefficients: 


;startloop 


IF (p.gt.0) ;skip this part if doing an MA process 
part=WH(0,p,2) 
part=2*{part-.5} ;now elements of part are between -1 and 1 
alpha=PARTAR(part,p) ;now alpha are stable AR coefficients 
ENDIF 
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26 IF(q.gt.0) ;8kip this part if doing an AR process 
27 part=WE(0,q,2) 

28 part=2*{part-.5} 

29 beta=PARTAR (part ,q) ;now beta are invertible MA coefficients 
30 ENDIF 

oLTs 

32 ;known 

aoe 

34 IF(p.gt.0) 

35 LABEL (alpha)=’ Autoregressive Coefficients’ 
36 POLYROOTS(alpha,1,p,100,rr,ri,ier) 

37 rmod={rr~2+ri~2}~.5 

38 LIST (alpha) 

39 LABEL(rmod)=’Modulus of Roots’ 

40 LIST(rr) 

41 LIST (ri) 

42 LIST (rmod) 

43 ENDIF 

44 IF(q.gt.0) 

45 LABEL (beta)=’Moving Average Coefficients’ 
46 POLYROOTS (beta,1,q,100,rr,ri,ier) 

47 rmod={rr~2+ri~2}~.5 

48 LIST (beta) 

49 LABEL (rmod)=’Modulus of Roots’ 

50 LIST (rr) 

51 LIST (ri) 

52 LIST (rmod) 

53 ENDIF 

54 ; 

55 3 Wow do correlations, spectra, and data: 
56; 

57 pq=p*q 

58 IF(pq.gt.0) 

59 rho=ARMACORR(alpha, beta,p,q,1,30,r0,ier) 
60 =ARMASP (alpha, beta,p,q,1,256) 

61 x=ARMADT (alpha, beta,p,q,1,0,200,30,rho,r0,ier) 
62 GOTO (display) 

63 ENDIF 

64 IF(q.gt.0) 

65 rho=MACORR(beta,q,1,30,r0) 

66 f=MASP(beta,q,1,256) 

67 x=MADT(beta,q,1,0,200) 

68 GOTO (display) 

69 ENDIF 

70 IF(p.gt.0) 

71 rho=ARCORR(alpha,p,1,30,r0,ier) 

72 f£=ARSP(alpha,p,1,256) 

73 x=ARDT(alpha,p,1,0,200,ier,r0) 

74 ENDIF : 

Thies 

76 ;display 

chp 

78 PAUSE 


79 zero=LIWE(30,0,0) 
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LABEL(rho)=’Correlogram, R(0)=@r0@’ 
LABEL(zero)=’ ” 

PLOT2 (rho, zero ,30,30,3,3,0,30,-1,1) 
part=CORRAR(rho,r0,30,rvar) 
part=ARPART (part ,30, ier) 
LABEL(part)=’Partial Correlogram’ 
PLOT2(part ,zero,30,30,3,3,0,30,-1,1) 
PLOTSP(f ,256,r0) 

PLOT (x, 200) 

GOTO(startloop) 


Example 2.8 | BAUER’s ALGORITHM 


The BAUER.MAC macro can be used to illustrate the structure of the mod- 


ified Cholesky decomposition of the correlation matrix of an MA(q) process. 
Using the entry point labeled ;known allows the user to use the macro for 
specified set of coefficients. 


omMmnNanh wnd 
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ae BAUER.MAC: macro to illustrate Bauer’s algorithm for 

wie finding MA parameters from correlations. 

oo The macro randomly generates MA coefficients, 

ae forms the correlations, and then finds and lists the 
5H modified Cholesky decomposition of the 

a correlation matrix of size M. 


aie INPUT: q, M (order of MA process and size of correlation matrix) 
a3 seed (random number generator seed for generating beta) 
PAUSE 

;start 

beta=WN (seed,q,2) ;U(O,1) 

beta=2*{beta-.5} ;U(-1,1) 

beta=PARTAR(beta,q) ;beta’s are MA coefficients 

; known ;use this entry point for known beta’s 
LABEL(beta)=’MA Coefficients’ 

rho=MACORR(beta,q,1,M,r0) ;get correlations 

gam=TOEPL (rho ,1,M) ;form matrix 

MCHOL(gam,M,L,D,ier) ; decompose 

L=TRAWS(L,10,10) ;we have to transpose L so that LIST works right 
LABEL(L)=’Lower Triangular Cholesky Factor’ 

M2=M*M 

LIST(L,M2,M,10f6.3) 

LIST (beta,q) 


To illustrate the macro, we used the ;known entry point with an MA(3) 


process having coefficients 0.8, —0.75, and —0.8, and obtained the output given 
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Table 2.3. An Example of Bauer’s Algorithm 


Lower Triangular Cholesky Factor 
1 | 1.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 
11 | .281 1.000 .000 .000 .000 .000 .000 .000 .000 .000 


21 | -.489 .455 1.000 .000 .000 .000 .000 .000 .000 .000 
31 | -.281 -.445 .579 1.000 .000 .000 .000 .000 .000 .000 
41 -000 -.306 -.633 .668 1.000 .000 .000 .000 .000 .000 


I 
| 
I 
51 | .000 .000 -.494 -.596 .727 1.000 .000 .000 .000 .000 
I 
| 
| 


61 -000 .000 .000 -.514 -.682 .767 1.000 .000 .000 .000 

71 -000 .000 .000 .000 -.637 -.656 .779 1.000 .000 .000 

81 -000 .000 .000 .000 .000 -.649 -.692 .786 1.000 .000 

91 | .000 .000 .000 .000 .000 .000 -.713 -.684 .775 1.000 
MA Coefficients 

so | - 800000 - .750000 - .800000 


in Table 2.3. Note how the diagonal elements of L are converging (slowly) to 
the corresponding elements of the beta array. 


Computational Problems 


C2.1. Use the MASP command with Q=256 to find the square modulus of the 
frequency transfer function of the 12th difference filter. The input to MASP will 
be that of an MA(12) with all coefficients zero except 312 = —1. Use PLOTSP 
to plot the logarithm of the square modulus. Does the result correspond to 
what we found in Section 2.3 on the effect of differencing? 


C2.2. Create a new macro called RW1 by modifying the RW macro to use 
U(-1,1) variables for the white noise process. Note that a realization from 
a time series process is also called a sample path. Do the sample paths for 
random walks using uniform white noise look much different than those using 
Gaussian white noise? Generate a plot of five sample paths of length 200 from 
each macro and compare them. 


C2.3. What four sets of MA(2) parameters (coefficients and error variance) 
can lead to the autocovariances R(0) = 1.3125, R(1) = 0.625, and R(2) = 0.25? 
Which of these sets leads to a characteristic polynomial having all of its zeros 
outside the unit circle? Which of these sets does TIMESLAB pick if you use 
the CORRMA command? (Hint: See the section on nonidentifiability of MA 
processes.) 
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C2.4. If X is an AR(1) process with a = 0.5, write X(t) as a function of 
X (t — 2) and some e’s and of X(t + 2) and some e’s. 


C2.5. Which of the following sets of coefficients can be regarded as the coeffi- 
cients of an AR process? 


a) a = (—0.90, 0.80) 
b) @ = (—1.54, 0.40) 
c) @ = (1.0, 0.904, —0.70) 
d) a = (—1.22, 1.16, —0.70) 


e) @ = (—0.09, 1.5884, —0.024, 0.90) 


C2.6. The macro ARGAMI.MAC will find the inverse of the (n x n) covariance 
matrix of an AR(p) process. Use it for n = 10 and for the AR(1) process having 
coefficient —0.5 and the AR(2) process having coefficients 0.3 and 0.9. From 
these results, infer that T=} is a banded matrix, that is, its (j, k)th element is 
zero for |j —k| > p. What else do you notice about the elements of the matrix? 


omrnanwnrh WBN FE 


sig ARGAMI.MAC: macro to find the inverse of the (n x n) Toeplitz 
B53 correlation matrix of an AR(p) process having 
es coefficients alpha and error variance 1. 

m5 INPUT: p, alpha, n (must be less than or equal to 10) 
PAUSE 

;start 

IF(n.le.10,s1) 

i) 

as n must be less than or equal to 10. 

>> 

GOTO (end) 

ssi 

rho=ARCORR(alpha,p,i,n,r0,ier) 


gam=TOEPL(rho,1,n) 

gami=MINV(gam,n, ier) 

n2=n*n 

LABEL(gami)=’Inverse of Corr. Matrix, R(0)=@r0@’ 
LIST (gami,n2,n,10f6.2) 

j;end 


C2.7. Write a macro that will form the (n x n) permutation matrix. 
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C2.8. Write a macro that will generate a realization of length n from an ARMA 
process whose errors have any of the eight distributions in the WN command. 
(Hint: Use zeros as starting values and use WN and FILT to get the MA part 
and then use the difference equation form of ARDT to generate a realization of 
length 2n. Use EXTRACT to extract the second half of this realization to use as 
the output realization of length n.) 


C2.9. Using the IDARMA macro in Example 2.6, find an AR, MA, and ARMA 
process of each of the following kinds: 


a) An excess of high frequency. 
b) An excess of low frequency. 
c) A single peak in the spectral density. 


Wait until you see a process having the desired characteristic and then 
reinvoke the macro with the ;known entry point and get a hard copy version 
of the plots for that process. 


C2.10. In this book, we consider primarily processes that can be expressed as 
a general linear process (see Section 2.5.9). Write a macro that will generate a 
realization of length n from the simple nonlinear process 


X(t) = e(t) = Be(t — 1)e(t — 2), 


where € is a white noise process. See pages 866-890 of Priestley (1981) for a 
general discussion of nonlinear time series. 


Theoretical Problems 
strait we Bh hee aie Phe, fetta 4H) banlint wad olf BOT 


T2.1. A time series X is said to be strictly stationary if for any positive 
integer n and any n+ 1 elements t;,...,t, and h of the index set T, the joint 
distributions of 


(Ens itt eee iaee (rete) aX Een) 


are the same. Show (a) that a strictly stationary series is covariance stationary 


and (b) that a covariance stationary Gaussian process is strictly stationary. 
(Hint: See Section A.4.) 


T2.2. Show that the sum and difference of two uncorrelated covariance sta- 
tionary time series are also covariance stationary. 
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T2.3. In Figure 2.5, how did we know that the five random walk realizations 
would probably fit on a vertical axis ranging from —30 to 30? 


T2.4. Show that the largest value that p(1) can have for an MA(1) process 
is .5. Show that for an MA(q) process, the largest value that p(1) can be is 
cos[7/(q + 2)]. Thus is there an MA(2) process having p(1) = 0.8? 


T2.5. Write the spectral density f of an MA(3) process as a third-degree 
polynomial in z = cosw, that is, as 


f(w) = 9 + 0,2 + 942” + 632° 


for some coefficients 6. 


T2.6. Show that if X ~ AR(p) and Y ~WN, and X and Y are uncorrelated, 
then the spectral density function of the series Z = X + Y is in the form of 
that of an ARMA(p, p) time series. See Granger and Morris (1976) and Engel 
(1984) for a general discussion of the properties of sums and products of ARMA 
processes. 


T2.7. Use the results of Problem T1.12 to show that an AR(2) process will 
appear cyclic of period p if aj = —2cos(2m/p) and ag = 1. (Hint: Show 
that an AR(2) process with these coefficients has a “signal to noise” ratio 
R(0)/o? that is very large, and thus the AR difference equation is almost a 
deterministic homogeneous difference equation in the sense that the variability 
in € is insignificant relative to that in X. See the ARCORR command in Section 
2.6 for a discussion of the relationship between R(0) and o?.) 


T2.8. We have defined the spectral density function on the interval [0,1]. 
Many authors define it on the interval [—7, 7] as in the following. Let R be the 
autocovariance function of a covariance stationary time series X and suppose 
the spectral density of X exists. Show that we can write 


us 


Fitts) = {Were ve Z, 
where 


fA) == S> R(ve™, NE [=a]. 


v=-—co 


T2.9. Prove part (c) of Theorem 2.3.1. 
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T2.10. Show that the following are true for integers j and k: 


max(j, 4) = VER +b By 
T2.11. Show that the Dirichlet kernel 
M ee 
Du(w) = De e2tiju 
j=-M 


discussed in Section 2.3 can be written as 


Z sin [(M@ + $)27w| . 


sin Tw 


Du(w) 
(Hint: Use a geometric series.) 


T2.12. Verify that the expressions for the coefficients of the bandpass filter 
example in Section 2.3 are correct. 


T2.13. Show that if P,, is the (n x n) permutation matrix, andT,, is an (nxn) 
symmetric Toeplitz matrix, then 


caSbALebeeand jet = PT, Pp: 


(Hint: Use the fact that P? =I,,.) 


T2.14. If € is a white noise process with variance o”, show that the process X 
defined by 


is a random walk process. 


T2.15. Suppose that X(0) is a random variable having mean px and variance 
o%. Fort > 0 let 
X(t) = —aX(t —1)+ €(t), 


where € ~ WN(o”) and ¢(¢) is uncorrelated with X(0) for all t. 
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a) Show that 


b) Thus show that 


E(X(t)) = (-a)"ux 


Cov(X(t), X(t + v)) = (—a)"**"92 +c? S(-a)+". 


j=0 
c) If |a| < 1, show that 
Jim E(X(t)) =0 
jim Cov(X(t), X(¢+v)) = celta 


A time series (such as this one) whose means and covariances converge as 
t > oo is said to be asymptotically covariance stationary. 


d) Show that if we let wx and o% be these limiting values of the mean 
and variance of X(t), then X is covariance stationary with mean zero and 
autocovariance function 2(-a)" 

o*(-—a 

Note that this device can be used to construct any autoregressive process 
as long as none of the zeros of the characteristic polynomial are on the unit 
circle. For an AR(p) process, we would assign arbitrary means and covariances 
to p initial random variables, determine the long-run mean (which will be 
zero) and covariance function, and then assign these “long-run” moments to 
the initial variables. 


T2.16. Show that a Gaussian AR(1) process is a Markov process; that is, for 
any positive integer n and any n + 1 integers 


ty < +: < ty < tay, 
the conditional distributions of X(t,41) given the X’s at all of the first n times 


are the same as those of X(tn41) given just X(t,). (Hint: See part (c) of 
Theorem A.4.2 and part (a) of Theorem 2.6.5.) 
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T2.17. Use long division to find the coefficients of the MA(co) representation 
of an ARMA(1,1) process having AR coefficient a and MA coefficient B; that 
is, divide 14+ Bz by 1+az. Do the division in such a way that only nonnegative 
powers of z are in the answer. What happens for large powers of z if la] > 1? 


T2.18. Let X and Y have the bivariate normal distribution with means both 
equal to zero, variances both equal to o?, and correlation coefficient equal to 
p. 


a) Show that 


1 1 
Rr Xa 0, Ye 0) re Fq arcsin p. 


(Hint: Use polar coordinates to do the integral of the joint pdf over the first 
quadrant.) 


b) Now suppose that X is a Gaussian stationary time series with mean 
zero, and define the clipped time series Y by 


1, if X(t)>0 
Y(t) = 
Oameit X(t) < 0. 


Use the result of part (a) to show that the autocorrelation function py of Y is 
related to the autocorrelation function px of X by 


2 ; 
py(v) = = arcsin px (v). 


CHAPTER 3 


Statistical Inference for Univariate 
Time Series 


In this chapter we describe some of the basic statistical inferences that 
can be made about univariate time series. In Section 3.1 we give the sampling 
properties of the descriptive statistics that were introduced in Chapter 1. In 
Section 3.2 we describe two standard tests for white noise. In Section 3.3 we 
discuss the problem of estimating the spectral density of a time series without 
making any assumptions about the model generating the data. This is called 
nonparametric spectral density estimation. In Sections 3.4 and 3.5 we consider 
respectively the questions of estimating the parameters of an ARMA model 
and identifying the orders of the model. Then in Section 3.6 we describe 
the so-called Box-Jenkins forecasting procedure and in Section 3.7 some other 
modeling strategies. In Section 3.8 we discuss the use of ARMA models for 
spectral density estimation, and finally in Section 3.9 we consider searching for 
periodicities in data. 


3.1. Sampling Properties of Descriptive Statistics 


In Chapter 1 we introduced the statistics 


t=1 
n—|v| 
Aly) == So (X()-XM(Xe+b)-*X), lel <a 
t=1 
i(v) = R(v) vj<n 
p(v) = R(0)’ Ju] < 


yer le we [0, 1], 


= 
Se 
\ 
sie 
wee 
san, 
Se 
S 
Di 
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as well as the sample partial autocorrelation function. We will now use upper 
case letters to represent time series observations, as they are considered to be 
random variables, whereas in Chapter 1 we were treating them primarily as 
sets of numbers. 

In this section we state a series of theorems summarizing the sampling 
properties (mean, variance, and asymptotic distribution) of X, R, p, and f. We 
will investigate the sampling properties of the sample partial autocorrelation 
function in Section 3.4.4 (see Theorem 3.4.6). 

Since the proofs of the theorems in this section are rather lengthy and 
widely available elsewhere (see Chapter 8 of Anderson (1971) or Chapter 5 
of Priestley (1981), for example), we will not provide them here. Rather, we 
attempt to give summaries of the implications of the theorems on analyzing 
data. 


3.1.1. The Sample Mean 


Theorem 3.1.1 | SAMPLING PROPERTIES OF X 


Let X be a covariance stationary time series with mean pt and autocovari- 
ance function R. Let X, = (X(1),...,X(n))? be a realization of length n 
from X and let X, = + )?_, X(t). Then 


a) E(Xn) = p. 


b) Var(X,) > 0 as n - 00 if and only if Cov(X(n), Xn) +0 asn > 00, 
a sufficient condition for which is that R(n) > 0 as n - 00. 


c) If 0° _., R(v) < co, then 
Jim, nVar(X,,) = Das Riv), 


vu=—-00 


which is equal to f(0) if X has a spectral density f and f is continuous at fre- 
quency zero, a sufficient condition for which is that R is absolutely summable. 


d) If X is a general linear process with independent, identically distributed 
errors and absolutely summable coefficients, then 


Vn(Xn — 4) + N(0, f(0)). 


Implications: Part (a) says that X,, is an unbiased estimator of p. A sufficient 
condition for an estimator to be consistent is that it is asymptotically unbiased 
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(that is, has expectation converging to the parameter being estimated) and 
has variance converging to zero. Thus part (b) gives a condition for X, to be 
consistent. A time series is also said to be ergodic in the mean if Var(X,,) — 0 as 
n —> 00, that is, if the average X, over time of a single realization approaches 
the average p = E(X(t)) at a single time t of the ensemble of all possible 
realizations. Thus X is ergodic in the mean if and only if as n gets large, 
adding new observations in the calculation of X, has no effect on it in the 
sense that Cov(X(n), X,) + 0. 

Part (c) gives two expressions that can be used to approximate Var(Xn) 
and also shows that Var(X,) goes to zero at the rate of 1/n. Finally, part (d) 
provides a general Central Limit Theorem for X,. This gives 


f(0) 


n 


iy 3 Za/2 


as a 100(1—a)% confidence interval for ». We will see in Section 3.3 that under 
the conditions of part (d), we can find an estimator f(0) such that 


X,+ Ze/2 ae 


is also a 100(1—a)% large sample confidence interval for p. 


Equivalent Number of Uncorrelated Observations 


To illustrate the results of Theorem 3.1.1, suppose that X is an AR(1) 
process with coefficient a, noise variance o?, and mean p. Then 


f(0) = 0? eg = eae 
PSF |1 an ae27 40/2 a (1 +a)?’ 
which we can write in terms of p = p(1) = —a as 
R(0)(1— p?) l+p 
f (0) = ———_— = R(0)-——__., 
(0) = Aare = R00) 


since ¢? = R(0)(1 — p”). Thus a 95% confidence interval for ys is given by 


ign HO Lee 
n 1l-—p 


This expression allows us to introduce the idea of an equivalent number of 
uncorrelated observations. Recall that a 95% confidence interval for the mean 
of a population having variance R(0) based on a random sample of size N is 


given by 
= | R(0) 
A. +.1.96 Ny 
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Thus to make sure that the confidence interval from the AR(1) process and 
the random sample have the same width, we would need 


Mics nee 
l+p 


For example, a sample of size n = 100 from an AR(1) process having p = .8 
is equivalent in this sense to a random sample of size 11 from a population 
having the same variance as that of the AR(1) process. On the other hand, if p 
is negative, there is actually more information about # in an AR(1) realization 
than in a random sample of the same size. In F igure 3.1 we illustrate the 
effect of ignoring autocorrelation in the calculation of confidence intervals for 
the mean of an autoregressive process. To obtain this figure, we generated 
100 realizations of length 100 for the process having a = —.5 (the macro used 
to create these graphs is described in Example 3.1). The confidence intervals 
which do not take autocorrelation into account fail to cover the true value of 
» (which is 0) far more often than could reasonably be expected. 

In general, a realization of length n from a time series having spectral 
density f is equivalent for estimating y to a random sample of size N from a 
population having the same variance R(0) as the time series, where 


N= nR(0) i n 


HO oe See 


vu=—0o 


Thus if f(0) is large (small), estimation of jy for the time series case is less 
(more) accurate than for the corresponding random sample case (see Problem 


C3.1). 
3.1.2. The Sample Autocovariances and Autocorrelations 


In Chapter 1 we stated that the sample autocovariance function 


n—|v| 


D2 (X(t) — X)(X(t+ lw) -X), ol <n, 


t=I 


A 1 
R(v) = — 
(vs) = - 
is generally accepted as the most satisfactory estimator of the autocovariance 
function R. The most popular alternative to R is the “unbiased” estimator 
n—|v 


| 
Ro) = > Yo (XW) - XX +0) - ¥) 


n a 


— tv); 


n—|o| 
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Figure 3.1. Confidence Intervals for » = 0 for an AR(1). 


so called because if we knew p and used it instead of X in the definition of 
R, the result would be unbiased, whereas R(v) with # replacing X would have 
bias —|v|/n. However, we rarely know p so neither R nor R is unbiased, and 
in fact their bias is small relative to their variability. Further, it is generally 
believed (see Example 3.2 and Priestley (1981), p. 324) that the mean square 
error of R(v) is smaller than that of R(v). 


The second reason for preferring R to R is that if we define R(v) = 0 and 
R(v ) = 0 for |v| > n, we have that R is a positive definite sequence whereas 
R need not be. This is an algebraic fact which we prove in the next theorem. 
For an example of where R is not positive definite, see Problem T3.6. 
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Theorem 3.1.2 | PositTivE DEFINITENESS OF R 


Let X(1),...,X(n) be a sequence of numbers and 


= Sl X (4) — X)(X(C+ fol) —X), olen 
R(v) =¢en 


0, |v] > n, 


where X =1+5~"_| X(t). Then the sequence R is positive definite if and only 
if X(1),...,X(n) are not all the same. 


Proof: We need to show that 
Py = Toepl(R(0),..., R(M — 1)) 
is positive definite for all 2 1 if and only if the X’s are not all the same. 
Let a, 
X(t)-X, t= Lhereceestatp 
Z(th= { 
0, 5S Ip 
Then the Z’s are not all zero if and only if the X’s are not all the same. Now 
it is easy to see that hi 
nT M= Za, 7 M, 
where Zy is the (n+M) x M matrix having first column zy = (Z(1),...,Z(n+ 


M))? with each successive column being the one before it shifted down one 
element with a zero placed on top. Thus I'y, is positive definite if and only if 


h? Zi, Zuh = (Zah)? (Zyh) = wiuwm >0 


for any nonzero (n + M)-dimensional vector h. But wiwo > 0 if and only if 
wa = Zyh F 0, that is, if and only if Zum is of full rank. Now let Z(j) be the 
first nonzero Z (which exists if and only if the X’s are not all the same). Then 
the jth through (M + j — 1)st rows of Zyy form a lower triangular (M x M) 
matrix having Z(j) on the diagonal. This matrix has nonzero determinant and 
thus these rows are linearly independent and Zy, is of full rank. 


In the rest of this section we will assume that # is known and has been 
subtracted from the data so that the time series from which we are sampling 
has mean zero. Thus we consider the properties of 


n—|v| 


Rw)== D> XOX(t+lo), — ol<n, 
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and note that these properties are a good approximation to the properties of 
the more realistic case where we must subtract X (see Anderson (1971), p. 
471). In finding the variance of R(v) we need to find E(R?(v)) which involves 
finding the expectation of the product of four X’s. 


Definition. A covariance stationary time series is called fourth-order station- 
ary if 
E(X(t)X(t + A)X(t+r)X(t+s)) 
is a function y(h,r,s) of h,r, and s only. The function 
K(h, r,s) = 7(h, r,s) — R(h)R(r — s) — R(r)R(h — s) — R(s)R(h — r) 
is called the fourth order cumulant function of X. 


If X is a Gaussian process, then 
y(h,r,s) = R(h)R(r — s) + R(r)R(A — s) + R(s)R(h—1r) 
(Isserlis (1918)), and so K(h,r,s) = 0. Some of the results in this section will 


have expressions including these cumulants but in practice we usually assume 
that they are zero, which they in fact are for a Gaussian process. 


Theorem 3.1.3 | PROPERTIES OF R AND p 


Let X be a zero mean, fourth-order stationary time series having fourth 
order cumulant function k, and let R and p be the sample autocovariance and 
autocorrelation functions of a realization of length n from X. Then 


a) E(R(v) — R(v)) = Hae) +0 asn—-oo. 


b) If all of the series involved converge, then 


Jim, nCov(R(h), R(g)) = So [R()R(r +h - 9) + R(r — 9) R(r +h) 


+ 3 K(h,—r,g—7r). 


r=—00 


If X has a continuous spectral density f, then 


1 
lim nCov(R(h), R(g)) = 2} cos 2thw cos 2mgw f? (w)dw 
noo 0 


SEC 3.1 DESCRIPTIVE STATISTICS 167 


if the series converges. If X is a general linear process with absolutel ly summable 
coefficients and independent errors having variance a? and finite fourth cumu- 
lant A4, then 


co 


emake G ks *4 R(h)R(a). 


r=-—00 


c) If X is a general linear process with absolutely summable coefficients 
and independent errors, then 


Jim, nCov(A(h), A(g9)) 


[e-e) 


= SF [ole +a)o(r +h) + ple — 9)o(r +h) — 2p(h)o(r)o(r +9) 


r=—oco 


— 2p(a)e(r)e(r + h) + 2p(9) p()p?(r)] 
2 z 2 
= aml [cos 2thw — p(h)] [cos 2rgu — p(9)| f?(w) dw. 


d) If X is a general linear process with absolutely summable coefficients 
and independent errors having variance o? and finite fourth cumulant A4, then 
any finite collection of R’s or p’s is asymptotically multivariate normal with 
asymptotic covariances given in parts (b) and (c). 


Implications: Part (a) says that the bias in R(v) is of the order 1 /n; that is, 
n times the bias converges to a constant. For fixed n, |v|/n is increasing with 
|v|, but R(v) should be decreasing with v so that the net effect should be that 
the bias is negligible as n increases. Note again if X is a Gaussian process that 
the cumulant terms in parts (b)—(d) will vanish. Note also that they do in part 
(c) for the asymptotic covariances of the sample autocorrelation function even 
if the process is not Gaussian. Parts (b) and (c) show the important fact that 
the elements of the covariance or correlation function are themselves correlated, 
which means that inferences made about them for different lags can not be done 
independently. The first expression in part (c) is called Bartlett’s formula, 
while the frequency domain formula in part (c) is due to Parzen (1961). We 
can get confidence intervals for R(v) or p(v) (see Example 3.3) from parts (b) 
and (c) since 
oo 
‘ A = 2 
jim nVar(R(v)) = }> [R2(r) + R(r — v)R(r + v)] 


r=—oo 
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while 


co 


jim, nVar(A(v)) = Y [02(r) + alr — vale + ») 


— 4p(v)p(r)p(r + ») + 2p"(v)p°(r)| 
2 2 2 
= ao |, [cos 2ruw — p(v)]° f?(w)dw. 


If X is a white noise series with variance R(0) = 0”, we have, approximately 
for large samples, that the elements of the covariance and correlation sequences 
are uncorrelated, and for v > 0, 


while R(0) ~ N(o?, 204/n). 


3.1.3. The Sample Spectral Density Function 


As in Section 3.1.2, we will assume that the mean of the process being 
considered is zero. Thus we consider the properties of 


ce 1 - mi(t—1)w ; 
fw) = —|)> xe? (-1)4[" we [0,1] 
t=1 


n-1 
= 3 Ripevertee’, 
v=—(n-1) 


Anderson (1971), p. 476 gives a heuristic argument that the properties of f 
with and without subtracting X are essentially the same. The next theorem 
verifies our observation in Chapter 1 that the sample spectral density function 
is too oscillatory to be useful for making statistical inferences. 
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Theorem 3.1.4 | PROPERTIES OF THE PERIODOGRAM 


Let X be a zero mean, fourth-order stationary time series having fourth- 
order cumulant function « and let f be the sample spectral density function 
for a realization of length n from X. Then 


a) If X has spectral density function f, then 


= (1- El) Re) cos 2ruw 


u=—(n-1) 


E(f(w)) 


/ eCaiere 


where F,, is the Féjer kernel 


F,(v) = NY (1 - eh cos 27uw = 7 (sean) 


n \ sin 7w 


b) If \yp~._49 R(v) cos 2muw < 00, then 


co 


Jim, E(f(w)) = SS R(v) cos 27vw, 


which is equal to f(w) if X has a spectral density f that is continuous at 
frequency w. 


c) If yy... |vR(v)| < 00, then R is absolutely summable and 


im, n{B(f(w)) ~ F(w)| =-2 3 vR(v) cos 2rvw. 


d) If X has a spectral density that is continuous at frequency w, and 


1 n—-1 


lim — d¢ |x(r, u, w)| = 0, 


r,u,w=—(n—-1) 


then 
Pw), £0, 5 


im, Vale) ={ 
BEteS 2f?(w), w=0,.5, 
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while if f is bounded in (w; — 6,w; +4) and (wz —6,w2 + 4) for some 6 > 0, 
we have 


Jim, Cov(f(w1), f(w2)) = 0, W1 Fw. 


e) If X is a general linear process having absolutely summable coefficients 
and independent, identically distributed errors, then for any integer M > 1 
and fixed frequencies w;,...,wm in (0,.5), we have that the random variables 


2f (wr) 2f (war) 
fei)’ f(wm) 


converge in distribution to the M independent random variables Q,...,Qm, 
each having the chi-square distribution with two degrees of freedom. Also the 
result continues to hold at all of the natural frequencies, that is, for the fre- 
quencies of the formw; = (j—1)/n. The asymptotic distributions of f(0)/f (0) 
and f(.5)/f(.5) are chi-square with one degree of freedom. 


f) If X is a normal white noise process with variance o”, then the peri- 
odogram ordinates, that is, the values of the sample spectral density at the 
natural frequencies, are independently distributed and 


Qf (w; 
2H) 43, ws #0,.5 
F(wj) ~x?, w; = 0,.5. 


o2 


Implications: Parts (a) and (b) show that the sample spectral density is 
asymptotically unbiased under general conditions, while part (c) shows that 
the bias is of order n. More importantly, these parts show that f is actually 
estimating an integrated weighted average of the values of f in a neighborhood 
of the frequency w. In Figure 3.2 we show the graph of the Féjer kernel F, 
for several values of n. This weight function is converging with n to a delta 
function, and thus unless the true spectral density has a very sharp peak in 
the neighborhood of w, the integral will converge to f(w). See Example 3.5 for 
the macro that was used to produce this figure. 

Part (d) shows that the sample spectral density function is not consistent 
since its variance does not go to zero. Further, the variance at frequency w 
is a constant independent of sample length. We illustrate this in Figure 3.3 
(see Example 3.4) where we have given the log periodogram of realizations of 
lengths 200, 400, 600, and 800 for a normal white noise series having variance 
1. Notice that the variability in these graphs is not decreasing with sample 
length. 
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Fejer Kernel For n=498 
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Figure 3.2. The Féjer Kernel for Various Values of n. 


The fact that the variance of f(w) is a function of f?(w), that is, the square 


of the mean of flw), is another reason for plotting the log of the periodogram 
instead of the periodogram itself, as it is easy to show (see Problem T3.3) that 


the limiting variance of the log of f(w) is a constant independent of w. 
Part (d) also shows that the values of the sample spectral density are 
asymptotically uncorrelated at different frequencies. Thus the log periodogram 
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Figure 3.3. Illustrating the Inconsistency of the Periodogram. 


appears to be a white noise series itself except that its level will change with 
the level of the true spectral density. We will see in the next section that 
we can make use of this uncorrelatedness to produce smooth versions of the 
sample spectral density that will in fact be consistent estimates of the spectral 
density. Parts (e) and (f) provide information on the asymptotic distribution 
of the sample spectral density. In essence, the sample spectral density has the 
chi-square distribution for large n in general, while if X is a Gaussian white 
noise series, then the periodogram is exactly chi-square for all n (see Problem 
T3.5). We will use this fact in the next section when we consider tests for white 
noise. 


3.2. Tests for White Noise 


In this section we consider two widely used methods for testing whether 
it is reasonable to conclude that a data set is a realization from a white noise 
time series. 


3.2.1. Bartlett’s Test 


Suppose that X(1),...,X(n) is a realization from a WN(c?) process. Thus 
the spectral distribution function F of X is given by F(w) = o7w for w € [0,1], 
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and we would expect that the cumulative periodogram 


Dja1 f(s) 
yer f(wj) 


of the data should be close to the points 


F (ux) = k=1,...,q¢ = [n/2]+1, 


Recall from Theorem 3.1.4 that except for w; = 0 and .5, the random variables 
27 (w;)/o? are asymptotically independent and identically distributed as x3 
random variables. But a x3 distributed random variable is the same as 2 times 
a variable having the exponential distribution with mean 1, and thus it can be 
shown that F(w;),...,F (w,) have approximately the same distribution as the 
ordered values Uj < --- < Uy of a sample of size q from a U(0,1) population. 
Thus we can use the well-known result (see Feller (1948), for example) that 


co 


Ue | < a) =e) lye ** 4. = Gla) 


j=—o0o 


lim P 
LB grea 


q-oco 


to judge whether a cumulative periodogram has a maximum deviation from 
the expected straight line too extreme to be reasonable under the hypothesis 
of white noise. This result is used in the WNTEST macro in Chapter 1. See 
Bartlett (1955), pp. 92-94 for more details about this procedure. 
The BARTTEST Command 
Given integers n and Q, and an array F of length g = [Q/ 2]+1, the command 
pval=BARTTEST(F,Q,n,B) 


will return 


B= yore gle — 5 
p=1-G(B) 
in the real scalar variables B and pval, respectively. The command 
pval=BARTTEST(B) 


will return pval= 1 — G(B) for a user-specified positive value B. In F igure 3.4 
we display the cdf F(b) = Pr(B < 5b), while in Table 3.1 we list the values of 
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1.007 of the Bartlett Statistic 
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Figure 3.4. Cumulative Distribution Function for the Bartlett Statistic. 


F (6) for b = .40,.41,...,2.00. The macro that generated the figure and table 
is discussed in Example 3.6. 


3.2.2. The Portmanteau Test 


Bartlett’s test is done in the frequency domain and uses the fact that the 
spectral density of white noise is constant. Another approach to the problem of 
testing for white noise is to use the fact that its autocorrelation coefficients for 
nonzero lag are all zero. The most popular method of this type is often called 
the portmanteau or Q test (see Box and Pierce (1970) and Ljung and Box 
(1978)). If X(1),...,X(n) is a sample realization from a white noise process, 
then the statistic 


a ee ~2(-) £. 2 
9 se) 2 emeseg oa akan 


Thus the hypothesis of white noise is rejected if Q > Se a Note that this test 
is often applied to the residuals (one step ahead forecast errors) from a model 
that has been fit to a data set. In this case, one degree of freedom is subtracted 
from m for each parameter that has been estimated. 


The QTEST Command 
The command 
pval=QTEST(rho,m,p,q,n,Q) 
will return the value Q of the test statistic and its p-value (in pval) for the 


errors of an ARMA(p,q) fit to a time series of length n. If QTEST is used for 
an original time series, then p and q should be entered as zero. 
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Table 3.1. Values of the CDF for the Bartlett Statistic 


ae 1.010 .741 1.020 .751 1.030 .761 
1.040 .770 1.050 .780 1.060 .789 1.070 .798 
1.080 .806 1.090 .814 1.100 .822 1.110 .830 
1.120 .837 1.130 .845 1.140 .851 1.150 .858 
1.160 .864 1.170 .871 1.180 .877 1.190 .882 
1.200 .888 1.210 .893 1.220 .898 1.230 .903 
1.240 .908 1.250 .912 1.260 .916 1.270 .921 
1.280 .925 1.290 .928 1.300 .932 1.310 .935 
1.320 .939 1.330 .942 1.340 .945 1.350 .948 
1.360 .951 1.370 .953 1.380 .956 1.390 .958 
1.400 .960 1.410 .962 1.420 .965 1.430 .967 
1.440 .968 1.450 .970 1.460 .972 1.470 .973 
1.480 .975 1.490 .976 1.500 .978 1.510 .979 
1.520 .980 1.530 .981 1.540 .983 1.550 .984 
1.560 .985 1.570 .986 1.580 .986 1.590 .987 
1.600 .988 1.610 .989 1.620 .989 1.630 .990 
1.640 .991 1.650 .991 1.660 .992 1.670 .992 
1.680 .993 1.690 .993 1.700 .994 1.710 .994 
1.720 .995 1.730 .995 1.740 .995 1.750 .996 
1.760 .996 1.770 .996 1.780 .996 1.790 .997 
1.800 .997 1.810 .997 1.820 .997 1.830 .998 
1.840 .998 1.850 .998 1.860 .998 1.870 .998 
1.880 .998 1.890 .998 1.900 .999 1.910 .999 
1.920 .999 1.930 .999 1.940 .999 1.950 .999 
1.960 .999 1.970 .999 1.980 .999 1.990 .999 
2.000 .999 
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The Choice of m in the Q Test 


One difficulty in using the QTEST command is in deciding what value of m to 
use. Asymptotically, this choice shouldn’t matter, but to investigate its effect 
in small and moderate samples, we give in Table 3.2 the results of a simple 
simulation study. For samples of size 50, 100, and 200, we generated 500 
Gaussian white noise series and counted how many times the null hypothesis 
of white noise was rejected for a =.01, .05, and .10 for m=5, 10, and 20. For 
each sample size, these should be 5, 25, and 50 for the respective values of a. 
For each a, we also counted the number of times out of 500 that the conclusion 
was not the same for the three values of m. From these results, it appears that 
(1) the true Type I error probability of the Q test tends to increase with m, 
particularly for smaller sample sizes, and (2) the effect of choosing m is more 
important as a increases. The reader may want to replicate this experiment 
to see if these patterns persist. The macros that were used to find the results 
are given in Example 3.7. Note that the QTEST and BARTTEST commands can 
be used in conjunction with the WNTEST macro discussed in Example 1.5. 


Table 3.2. Results of a Simulation Study of the Effect of the Choice of m in 
the Q Test 


alpha=.01 alpha=.05 alpha=.10 
n=50 n=100 n=200 n=50 n=100 n=200 n=50 n=100 n=200 
m=5 8 8 6 31 27 27, 46 52 35 
m=10 14 6 4 31 26 25 55 56 54 
m=20 18 14 6 46 43 23 72 70 $3 
# diff. 21 20 11 57 48 45 81 82 74 


3.3. Nonparametric Spectral Density Estimation 


We saw in Section 3.1.3 that the sample spectral density function y is an 
inconsistent estimator of the spectral density function f. In this section we 
describe how f can be modified to produce consistent estimators. In Section 
3.8 we will use ARMA models to obtain other consistent estimators, but for 
now we will not assume that X follows any particular model. Thus we call the 
resulting estimation procedures nonparametric in analogy with nonparametric 
statistics where no parametric assumptions are made about the distribution of 
a data set. We will see in Section 3.8 that if the true spectral density being 
estimated is that of an invertible ARMA model, then using parametric methods 
will provide spectral estimates having better properties than the nonparametric 
methods. 
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3.3.1. Smoothing the Sample Spectral Density 


The basic problem with f is that it is too wiggly to be an adequate esti- 
mator of a function that is typically smooth over much of its domain. In this 
section we consider a simple averaging approach to smoothing the periodogram. 
In Section 3.3.2 we will consider more elaborate smoothing methods. We will 
present the simple averaging in some detail as it illustrates the general ideas 
of smoothing and is not as difficult mathematically as the methods in Section 
3.3.2: 


Recall from part (e) of Theorem 3.1.4 that under general conditions 


Mitcey 2 2([n/2] 
f0), 4)... 42) 
are asymptotically independent random variables with 


: ip) x3; 


W5 oe 0, 9) 
f(wj) ~ 


f (w;) x7, w; =0,.5: 


Consider estimating f at one of the natural frequencies w; = (j — 1)/n by 


Sh 


icky 


(wj+e), 


that is, by the average of f (w;) and the m values of the periodogram on either 
side of it. If wj;m <0 or wj4m > .5, we can use the fact that i is symmetric 
about 0 and .5 to obtain the elements in the sum. 

In Example 3.8 we give a macro called AVEPER that can be used to do 
this simple averaging. To illustrate the procedure, consider Figure 3.5 which 
contains four graphs. The first is the log of the standardized form of the true 
spectral density of the AR(5) process: 


X(t) + 1.7X(t — 1) + 2.4X(t — 2) + 1.634X(t — 3) + 0.872X(t — 4) 
+ 0.168X(t — 5) = €(t), 


where € is white noise with variance 1. We then give the averaged periodogram 
for a realization of length 200 for m = 1, 4, and 7. As m increases, the estimate 
becomes smoother (less variable), but the bias in the neighborhood of the peak 
becomes quite pronounced. This is to be expected as in this neighborhood the 
periodogram is changing rapidly. For the frequency where the peak occurs, the 
final estimate is the average of the periodogram at that frequency and the m 
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Figure 3.5. The Effect of Increasing m in Averaging the Periodogram. 


frequencies on either side of it. Most of these values of the periodogram are 
much smaller than the value that is being estimated. 

Now suppose that wj-m > 0 and wj4m < .5. We can think of f(w;) 
as averaging all of the periodogram values in the frequency band w; + m/n, 
that is, as a smoother having bandwidth m/n. As in the kernel probability 
density estimator in Section 1.7, we consider bandwidth to be half the width 
of the frequency band, that is, the width of the interval on each side of the 
center frequency. Arguing heuristically, we assume that n is large and use the 
asymptotic properties of f to obtain 


E(f(w;)) = <i > F(wj+k) 
k=-—m 


5 l pit eee 1 * 
Var(f(w;)) = Gen) Do Plein) S sy _ max, P (wise). 
k=—m e = 


These two expressions exhibit clearly the importance of the smoothness of f, 
as well as the tradeoff between variance and bias in f(w;) in the choice of m. 
For a fixed value of n, increasing m will increase the bandwidth and lead to 
including values of f at frequencies farther from the target frequency w; in the 
expected value of f(w;), thus increasing the possibility of a larger bias. If f is 
smooth in the neighborhood of w;, then this bias will not increase appreciably. 
On the other hand, increasing m has the effect of decreasing the variance of the 
estimator, again assuming that f does not significantly increase as frequencies 
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get farther from w;. Note that the simplest case is when X is white noise, in 
which case f is constant over (0,.5], and we can increase the bandwidth without 
increasing the bias while decreasing the variance at the rate of 1 /(2m+1). At 
the other extreme, we might be estimating f at a frequency that is adjacent 
to a band where f rises sharply to a peak, in which case increasing m could 
increase both the bias and the variance. This phenomenon is known as leakage 
as high power in f in one frequency band “leaks” into estimates of f at other 
frequencies. 

As long as f is bounded, we can let m go to oo as n goes to oo so that the 
variance of f (w;) goes to zero. As m and n get large, we have 


E(f(w;)) = <i SD f(wj+n) = = f(w)dw. 
k=-m 


2m w3—m/n 


If we let m/n get small and-assume that f is such that it is roughly constant 
over a small interval, then the above integral is approximately ae f(w;), and 
i (w;) is asymptotically unbiased. 

For two different fixed frequencies w; # w,, we have that f (w;) and f(w,) 
will become independent as m/n goes to zero since they will eventually be 
based upon averaging values of the periodogram for nonoverlapping frequency 
bands. 

We next consider the asymptotic distribution of ‘ff (w;). For wm > 0 and 
Wjim < .0, we have 


a 1 a) ; 
f(wj) ® TT ia Fe Os, 
k 


=-m 


where © means has the same distribution as, and the (’s are independent x? 
distributed random variables. We know that a pure sum of independent x? 
random variables has itself a y? distribution with degrees of freedom equal to 
the sum of the individual degrees of freedom. Thus if X ~WN (07), we would 
have that f(w;) ~ {o?/[2(2m + 1)}} Xtlantis: The distribution of a general 
linear combination of y? random variables is often approximated by that of 
Q = cx?, where c and v are chosen so that the mean and variance of Q are the 
same as those of the linear combination (see Tukey (1949)). We will use this 
device to approximate the distribution of f(w;)/ f(w;). Thus we have 


B(Q) = w = B(L42) oie 


f(w;) 
2 Ff (w;) ee 1 
Var(Q) = 2c*v = Var( Fy) Suit, 
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We then set cv = 1 and 2c?v = 1/2(2m +1) and obtain c = 1/v and 
v = 2(2m + 1), which gives 


vf(wj) 


f(w;) Xv» W5 # 0, 5. 


Note that in this simple averaging case we could have obtained this immediately 
from our assumption that f is constant around w;. 


Estimating f(0) and f(.5) 
The arguments given above must be modified when we consider estimating 


f at frequency 0 or .5, as then the values of f on each side of the center 
frequency are the same. If we write, for example, 


10> ws [A f(0) +2 Li jes]. 


we find 


2 
~ Xy/2) Ws; ==0. 0; 
9 f (w;) /2 J 


3.3.2. Kernels and Fourier Series Approximations 


The next natural step in estimating f is to insert a weight function into the 
averaging of the periodogram so that values of f at frequencies far removed 
from w; get less weight than values at frequencies close to w;. We consider 
integrated weighted averages of f of the form 


es » Rig Afar 


where K is a weight function called a (spectral) window. For a fixed frequency 
w, this expression essentially says to superimpose the weight function onto the 
graph of the periodogram (with its value at zero centered at frequency w), 
then multiply the two functions together (which down-weights values of f far 
removed from the “center” frequency w), and finally do the integral. To find 
the estimate at a different frequency, the same process is carried out with the 
weight function moved over to be centered at the new frequency. Thus the 
function K essentially dictates what part of f can be “seen” when finding the 
estimate at a certain frequency. This is the origin of the term window. We 
will study windows of various shapes and discuss the effect of their width (see 
Figure 3.7 for some examples of windows). 

Using spectral windows then is a natural extension of the idea of averaging 
the sample spectral density. We could use weighted averages that are sums 


SEC 3.3 SPECTRAL DENSITY ESTIMATION 181 


instead of integrals, but the integrated averages arise naturally from another 
point of view, which we now describe. Such averages arise naturally from 
considerations of the general theory of the Fourier series representation of a 
function on a finite interval (see Section A.2). An expression of the form 


co 


f(w) = Sy R(v)e~27# 


vu=—-co 


where : 
R(v) 2) f(w)e?™* dus 
0 


is called the Fourier series representation of f, and the coefficients R(v) are 
called its Fourier coefficients. The periodogram 


f(w) = 3 R(v)e727* 


v=—(n-1) 
is thus actually an estimator of the nth partial sum 


n—-1 


fn(w) = >> Riven aues 


uv=—(n-1) 


From this it is clear that there are two sources of error in f as an estimator 
of f: a truncation error due to using f, to approximate f, and an estimation 
error due to having to estimate R. In the general theory of Fourier series it 
is well known that the approximation of some functions by their Fourier series 
can be improved by applying a sequence of weights k,, (v) to R in f, (see the 
bandpass filter example in Section 2.3, for example), giving 


n-1 


fn,k(w) = 7 ke (v) R(v)e7 27 


v=—(n—-1) 


n—-1 
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The sequence k,, is called a lag window, and the function 


K,(w) = = kn(v)e~2"", = w E (—00, 00), 
v=—(n-1) 


is called a spectral window. Thus the integrated averages that we have been 
discussing arise naturally from the idea of applying weights to the sample 
autocovariances. The weights k,(v) are usually an even function of v and 
thus K, is real and symmetric about .5. Further, K,(w) is defined for all 
w € (—oo, co) and is periodic of period 1. This also means that K,, is symmetric 
about zero (see Problem T3.10). 

Note that k, and K,, are Fourier pairs since 


al 1 
i Ky (w)e2"¥ = | ( 2. kn (u)e~ 285) e215 dy 


1 
1S kn (v) il e2™(v-J)u qu 
0 


v=—(n-1) 
=) PGR 


This suggests the following definition. 


Definition. A spectral estimator of the form 


fax(w) = Ss kn (v) R(v)e7 272 


v=—(n-1) 


= i ei aay 


is called a window estimator with lag window k,, and spectral window K,,. If 
kn(v) = 0 for |v| > M for some integer M (called a truncation point), we say 
that the estimator is of truncated form. 


Note that the sample spectral density function itself is of the above form 
with k,(v) identically one, which gives that K, is the Dirichlet kernel that we 
saw in Section 2.3 (see Figure 2.3). We will assume the following throughout 
this section: 


1. kn(0)= 1. 
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2: Je K?(w)dw < oo. 
3. For any € > 0, Ky(w) > 0 uniformly as n — 00 for |w| >. 


4. The weights k,(v) decay to zero fast enough that 


n—-1 


HO) 
v=—(n-1) 
lim “==! Uo! 
n—0o n—-1 
dD Fale) 
v=—(n-1) 


Windows of Scale Parameter Form 
A variety of weight functions have been suggested, most of the scale pa- 


rameter form, that is, 


for some integer M < n called the scale parameter, with \ being a function 
satisfying 


Po alO) =i 
2. A(—u) = A(u). 


3. f°. A?(u)du < co. 


Definition. The function defined above is called a lag window generator, 
while the function 


1 co 


2g oe 


A(w) 


A(u)ei” du, w € (—oo, oo), 
is called the spectral window generator corresponding to X. 


For lag windows of scale parameter form, it can be shown (see Parzen 
(1963a)) for large M that 


Kn(w) = 2nMA(27Mw). 


In Table 3.3 we give information about eight of the lag and spectral window 
generators that are commonly used. The last five columns in the table con- 
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tain numerical quantities that are important for judging the adequacy of the 
window. We describe these quantities below. 

The first five lag window generators are zero outside of |u| < 1 and thus 
kn(v) =0 for |v] > M and f,,, only involves values of R(v) for |u| < M, and M 
is then called a truncation point. This terminology is often used even when A 
is not of truncated form. The last window was originally considered by Parzen 
(1958) and has been discussed by Cogburn and Davis (1974). 

In Figure 3.6 (see Example 3.9), we display each of the spectral windows 
K,, for n = 500 and M = 20. For the first five windows, K,, is only a function 
of M. We plot each window in two parts, the first part being for frequencies 
between —.1 and .1, and the second part for frequencies between .1 and .5. The 
basic feature of the windows is that they decay rapidly from frequency zero 
and then rise back up again in what are called sidelobes. All of the windows 
converge to a delta function as M increases. This is analogous to letting the 
bandwidth of the simple averaging smoother of the previous section go to zero. 
The truncated periodogram, Tukey, and Parzen-Cogburn-Davis windows are 
negative in certain frequency bands, while the others are nonnegative. This is 
important because using a negative spectral window can result in a spectral 
estimator that is negative. 

Since the main lobe of a window is not rectangular, it is difficult to measure 
how wide it is. In the last column of Table 3.3 we give what is called Parzen’s 
measure of the bandwidth of a window, denoted Bp and defined to be the 
width of a rectangular window having the same value at w = 0 as K,, and the 
same area as K,,. Since the areas of all of the K, are one, we have 

1 el 
gs 2nMA(0)  Kn(0)’ 
Notice how the relative appearance of the windows follows the values of Bp. 
For example, the Parzen window for M = 20 is wider than the Tukey window 
for M = 20. 

In Figure 3.7, we have superimposed the Parzen and Tukey windows for 
truncation points 24 and 18, respectively. Note how the main lobes seem to 
line up, which could be predicted based on the bandwidths of the two windows. 


The WINDOW Command 
The command 
f=WINDOW(rho ,RO,M,Q,ioptwL,n,c]) 


will calculate the nonparametric spectral density estimator at the Q frequencies 
w; = (j —1)/Q for j = 1,...,Q and store the first [Q/2]+ 1 of them in the array 
f. Input is M (the scale parameter to use), the sample variance RO, the number 
of frequencies Q, the integer ioptw which determines which window is to be 
used (ioptw is 1 through 8 corresponding to the eight windows in Table 3.3), 
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Table 3.3. Some Window Generators and Their Characteristics 


A(w) = = fj A(uje™’du g AM SP A?(u)du Emin Bp 
te ee 
Generators of Truncated Form, i.e., A(u) = 0, Jul > 1 
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Parzen Window: X(u) = ve Saas ee 1 
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Generators Not of Truncated Form 
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TU 
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Window 1, n=500, M29 
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Window 1, n=508, 28 
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Window 4, n=508, 29 
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Figure 3.6. Eight Spectral Windows for M = 20. 
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Figure 3.6 (continued) 
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the array rho which contains the sample autocorrelations to be used (M of them 
for the first five windows and n-1 of them for the last three). If ioptw is 6, 7, 
or 8, then the sample size n must also be input. The argument c is an output 
real variable and is used to find confidence intervals for the spectral density. 
We describe this further below Theorem 3.3.1. 


3.3.3. Sampling Properties of Estimators 


As we have pointed out, the properties of a spectral estimator based on 
smoothing f will depend on the smoothness of f and the properties of the 
weighting function being used. In the case of a lag window of scale param- 
eter form, the properties of the weighting function will depend on the scale 
parameter M and the lag window generator \. 
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29.007 3rzen (WE24) and Tukey (¥E18) Windows 


0.0 0.@ 0.04 0.% 0.8 0.18 


0.16 Parzen (M24) and Tukey (M618) Windows 


0.18 0.18 0.26 0.34 0.42 0.50 


Figure 3.7. The Parzen and Tukey Spectral Windows for Comparable Band- 
widths. 


Measuring the Smoothness of f 


In general, the smoothness of f can be measured by how its derivatives 
behave. For example, if f has a large peak at frequency w, then f’’(w) will be 
very large in the region near frequency w (see Problem C3.13). Thus we can 
index the smoothness of f by the number of finite derivatives that it has. 


Definition. A spectral density f is said to be p-differentiable if 


S> lvPIR(v)| < 00. 


v=-O 
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The generalized pth derivative of f is given by 


fPl(w) = Me, |vPR(v)e~ 27, 


uvu=—o0o 


Note that if f is p-differentiable, then the pth derivative does in fact exist 
since, for example, 


co 
= ys |R(v)27v sin 2rvw| 


v=-—00 


—— R(v) cos 2ruw 
w 


e-e) 
=2n $5 |o||R(v)||sin 2rvw| 


v=—00 


S$ 2m YP PrIIR(e)| 


v=—0o 


< OO. 


Note also that the spectral density of an ARMA process is p-differentiable for 
all integers p (see Problem T3.8). 


We should be able to adequately smooth f using some truncation point 
and lag window generator 4 as long as f doesn’t have a peak so sharp relative 
to the width of X that the smoothing won’t work. 


Measuring the Smoothness of \ 


The character of A can be measured by what is called its characteristic 
exponent and characteristic value. 


Definition. The characteristic exponent q of the lag window generator X is 
the largest integer k such that 


.- 1—X(u) 
Se al? 


exists, Is finite, and is nonzero. If such a q exists, then the characteristic value 
of X is given by 
. 1-A(u 
NO) = tim 2A) | 
u+0 ful? 
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If the limit exists and is finite for all integers k, then we say that q = co 


Note that gq measures how sharply \ can fall off as w varies away from 
zero; that is, it measures the curvature of A in the neighborhood of zero. For 
example, if g = 2, then 1 — \(u) behaves like a quadratic function of u near 
zero. Note from Table 3.3 that the truncated periodogram window has q = oo, 
the Bartlett window has q = 1, the next five windows have gq = 2, and finally 
the Parzen-Cogburn-Davis window has q = 2r. 

The next theorem summarizes a large number of basic results about the 
spectral estimators introduced in this section. The theorem is primarily con- 
cerned with estimators of scale parameter form. Part (g) allows us to make 
inferences about f for spectral windows of general form. Complete descriptions 
of these results can be found in Anderson (1971), Chapter 9 or Priestley (1981), 
Chapter 6. Many of the results of the theorem were found originally by Parzen 
(1957a, 1957b, 1958). After the statement of the theorem we describe how the 
results are used in practice. 


Theorem 3.3.1 | PROPERTIES OF WINDOW ESTIMATORS 


Let X(1),...,X(n) be a sample realization from a covariance stationary 
time series X which has spectral density function f which is p-differentiable for 
some p > 1, and let \ be a lag window generator having characteristic exponent 
q > 0 and characteristic value \{%. Let 


fam (w eat X(— ap) Re —2nrivw 


v=>-0o 


where M is chosen as a function of n so that M > «© asn oo. Note that the 
limits on the sum are actually —M to M if X is of truncated form and —(n—1) 
to n— 1 otherwise. Then 


a) The estimator fa,m(w) is asymptotically unbiased, that is, 
Jim B [fa,ae() — f(e)] = 0. 
b) AsyMPToTIc B1as. The rate of decay of bias in fy 44 (w) is determined 
by: 


i) If p > gq and M is chosen so that M%/n — 0 for p > 1 or 
M2*1~-? /n -+ 0 for p < 1, then 


lim M? [E(f,,xe(¥)) — f(w)] = -X fle). 


n->co 
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li) If p < q and M is chosen so that M?/n - 0 for p> 1 or M/n-+ 0 
for p < 1, then 


Jim M? [E(frac(«)) - f@)] =0. 


c) ASYMPTOTIC VARIANCE. If X is stationary to fourth order with abso- 
lutely summable autocovariances and fourth-order cumulant function, and M 
is chosen so that M + co and M/n - 0 as n - oo, then the rate of decay of 
the variance of f, (w) is determined by 
2f?(w) [. A*(u)du, w=0,.5 


—-co 


lim a7 Var faa (w)) = 
n-+00 f?(w) ie r?(u)du, w #0,.5 


Jim, 5g Covlfa.at wr), Arne (2) =0, wi Fup. 


d) ASYMPTOTIC MEAN SQUARE ERROR. If p > q as in part (b), and the 
assumptions in part (c) are valid, then for w # 0,.5, the rate of decay in the 
mean square error in fy y(w) as an estimator of f(w) is determined by 


MSE(fs,a1(W)) = E [fa.ar(w) — far] = Var(fa,re(u)) + [BUA ac(e)) - fw) 


= ~f(w) ia \?(u)du + uta” fl (w))?, 


both terms of which will go to zero at the same rate if M is chosen as M = 
cn/24+1 for some constant c, in which case 


n—-+co 


lim n?4/(24+)) MSE( fy ¢(w)) = cf?(w) / ii d?(u)du+ a) fil)? 


and 
@ fiw) 


c 


lim 4/24) [B (A, u(w)) — f(e)] = 


n-—>co 


For fixed characteristic exponent q, the minimum value of the mean square 
error as a function of M for a lag window generator X is proportional to 


1 
Emin = Geanls | r?(u) du. 
0 


e) If X is a linear process with absolutely summable coefficients and inde- 
pendent, identically distributed errors, if M -+ 00 and M/n > co asn 3 Co, 
and we let 


f, = (fr,m(w1),---,fraa(wr))” and f, = (f(w1),-.., F(we))® 
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for r fixed frequencies w1,...,w,, then 


Vuh ~&) 9 Ne(Or,V); 


where the elements of V are given in part (c). 


f) If the conditions of parts (b) and (e) hold, if limp... n/M??t! < 00 for 
p<qorn/M7%+1 -5 0 asn > o for q < p, and if f(w) > 0, then 


los ful nloey re Vz jog acl) £.y (0, [ pt 2?(u)du 


g) For a general spectral estimator of the form 


n—-1 


facnl@) Sad eek) A g)lag ae 


v=—(n-1) 


we have that the random variable v fn (w) /f(w) is approximately x2, where 
the equivalent degrees of freedom v are given by 


. 2n 
= le a ee 
pela 1) k3 (v) 
If k, is of scale parameter form with lag window generator A, then 


2n 


Y= Mf, 2(u)du’ 


Implications: There are three kinds of facts in this very complicated theorem. 
First, parts (a) and (b) show that f) 14(w) is asymptotically unbiased, while 
part (c) shows that its variance goes to zero if M/n — 0 as n > oo, which 
together give that f,,(w) is a consistent estimator of f(w). Part (c) also shows 
that estimators at fixed unequal frequencies are asymptotically uncorrelated. 
Second, part (d) and the other information in parts (b) and (c) allow us to 
make statements about how rapidly fxm(w) converges to f(w) and also to 
compare lag window generators \ and scale parameters M. Parts (b) and (c) 
show that the bias and variance behave inversely as a function of M, with the 
bias being proportional to M~? in the usual case where p > q; that is, the 
spectral density is smoother than the window, and the variance is proportional 
to M. Thus a smaller M gives a less variable estimator, but one having more 
bias. 
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For fixed M and q, the bias is proportional to the characteristic value \(Y 
of the window being used, while the variance is proportional to ihn A?(u)du. For 
the windows in Table 3.3 having g = 2, there is no window that has minimum 
values for both of these characteristics, and in this sense there is no optimal 
window. Since frm (w) is in fact a biased estimator, a true measure of its 
performance as an estimator of f(w) is its average squared distance from f(w), 
that is, its mean square error which is given in part (d). In this part we see that 
for all possible windows having characteristic value g, the bias and expected 
mean square error go to zero at the rates of n~9/(29+1) and n-24/(29+1). For 
q = 2 this means choosing M to be proportional to n!/5 (see the related issue 
of choosing rbins in the DENSITY command), and the rates of decay of bias and 
mean square error are n~?/5 and n-4/5. This last result is often expressed as 
the “four-fifths rule”. These considerations argue for windows having a larger 
value of g. This rules out the Bartlett window as it has q = 1. The second 
part of part (d) gives us a means of comparing windows for a fixed value of 
q by comparing their values of Emin. In this sense the Bohman window is 
optimal. There are a wide variety of other methods for comparing windows 
(see Chapters 6 and 7 of Priestley (1981)) but we recommend using either the 
Parzen or Bohman windows as they are nonnegative, are of truncated form so 
that one need only calculate M correlations, and have good Ey, values. The 
remaining problem is the choice of M (see below). 

The third kind of facts in the theorem are those about the asymptotic 
distribution of Te m(w) given in parts (e)-(g). These allow us to find confi- 
dence intervals for f(w) at an individual frequency w. Part (f) also shows that 
log f,,m(w) is approximately N (log f(w), * J, A?(u)du); that is, the variance 
of log fr, M(w) is approximately constant over frequency. Again this motivates 
plotting not f,,4(w) but log frm (w). 

The confidence intervals based on parts (e), (f), and (g) are given by 


fi.m(w) + Za/2\| viknle) [ A?(u)du, 


(<frarlwo), chs an()), 


where c = exp(Zq/2\/“ f°. d2(u)du), and 


[ee “fate 


2 oer 
Xa /2 X1-a/2 


respectively. In the WINDOW command the number returned in the argument 
¢ (if it is included) is the value of c in the confidence interval formula above. 
Thus to display an estimator and the confidence intervals at each frequency 
one can use: 
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spec=WINDOW(rho,RO,M,Q,ioptw,n,c) 
specl=spec/c 
specu=spec*c 


PLOTSP(spec,Q,RO,specl,Q,RO,specu,Q,RO) 


These intervals are for f at a single frequency. A large sample simultaneous 
confidence band on f for all frequencies can be obtained by (see Woodroofe 
and Van Ness (1967)) multiplying both terms in the confidence interval based 
on part (f) by exp((2log M)'/?). 


The Choice of M 


The choice of M determines the amount of smoothing done, with too 
large (small) a value resulting in undersmoothing (oversmoothing). Wahba 
(1980) discusses objectively choosing the amount of smoothing to do in a sim- 
ilar setting (estimating the logarithm of the spectral density using essentially 
windows of the Parzen-Cogburn-Davis form), but in general the choice of M 
is very difficult unless some information about the function being estimated 
is known. Basically, if f has a narrow peak, we would like the bandwidth of 
the spectral window to be narrow so that leakage doesn’t occur. This however 
leads to undersmoothing in other frequencies. Thus nonparametric spectral 
estimators have trouble “resolving” peaks without introducing spurious peaks 
in other ranges. We will see later that parametric spectral estimators can solve 
this problem in many cases. In any event, the prevailing view on the choice of 
M is to try more than one value and use the resulting plots to make general 
statements about f. For example, Parzen (1967) suggests using three values, 
M,, M2, and Ms, where 


In Figure 3.8 (see Example 3.10), we give spectral estimates using the Parzen 
window for three data sets: (1) the critical radio frequencies data, (2) the 
monthly sunspot data, and (3) the magnitude of a star data. Note that the 
estimator with the most variation in each case is the one with the highest trun- 
cation point. Notice how similar the results are for the critical radio frequencies 
data and the monthly sunspot numbers except that the sunspot numbers do 
not have the peak at period 12 months that the critical radio frequencies have. 
Thus the critical radio frequencies data are probably being influenced by an- 
nual weather variables such as temperature. The low frequency peak in these 
two series is in fact the so-called sunspot cycle. 
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Figure 3.8. Parzen Window Estimates for Three Series, Each Using Three 
Different Truncation Points. 
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3.4. Estimating the Parameters of ARMA Models 


In Section 2.5.5 we obtained the spectral density function of an ARMA 
process, while in Section 2.6.1 we showed how to forecast future values of the 
process if we knew the orders of the process and its parameters. Given data 
X(1),...,X(n) from an ARMA process of orders (p,q), we would like to find 
estimates &, B, and G? of the parameters of the model as this would allow us 


to estimate f by 
q 
p By e2tikw 


fw) = 6? 9 1, 


P 
y a; 2nijw 
ajze 


and we could substitute the parameter estimates into the formulas for forecast- 
ing future values of X. In this section we assume that the orders p and q are 
known and discuss the estimation of the parameters. In the next section we 
consider the problem of determining p and q. 


2 


3.4.1. Maximum Likelihood Estimation 


The first method that we consider is the application of the general principle 
of maximum likelihood. If we assume that the data are from a Gaussian ARMA 
process, we saw in Section 2.6 how the ARMA likelihood can be evaluated for 
specified values of its parameters via the Kalman Filter Algorithm. Much 
of the recent emphasis in calculating the maximum likelihood estimates for 
ARMA processes has been on combining a likelihood “evaluator” (see Mélard 
(1984)) with a nonlinear “optimizer” that requires no derivatives. This is in fact 
what the TIMESLAB command DTARMA does using the Nelder-Mead simplex 
algorithm as the optimizer (see O’Neill (1971)). 


The DTARMA Command 
The command 
DTARMA(x,n,p,q,maxit,eps,alpha, beta,rvar,m211,ier,del) 


will attempt to find the maximum likelihood estimators for an ARMA(p, q) 
process given a realization of length n. The input integers n, p, and q contain 
n, p, and q, respectively, while the input arrays x, alpha, and beta contain the 
data and starting values for the coefficients. These starting values can be ob- 
tained from a variety of commands, including (1) method of moments estimates 
(see Section 3.4.3), (2) the SEASEST command (see Section 3.4.2), or (3) the 
ARMASEL command (see Section 3.5.3). In some cases, entering arrays of zeros 
as starting values will also lead to convergence of the likelihood maximization 
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procedure. The Nelder-Mead optimization procedure maximizes the likelihood 
with respect to p+ q variables, namely the elements of a and B. At each step, 
the algorithm calculates the likelihood at p+q points, namely at the vertices of 
a simplex, that is, a (p + q)-dimensional polygon. Then the algorithm tries to 
determine if there is a relative maximum of the function within this simplex. 
If so, then the simplex is made smaller and the procedure is done again. If not, 
then the simplex is moved and the procedure retried. At each step, the sample 
variance of the values of the likelihood function at the vertices of the simplex 
is calculated, and convergence is assumed if this variance is smaller than the 
convergence criterion that the user entered in the real scalar eps. The input 
integer maxit is the maximum number of evaluations of the likelihood function 
that the user is willing to allow; that is, maxit is p+ q times the number of 
steps of the algorithm that are to be allowed. 

The quantities that are returned by DTARMA depend on what is returned in 
the integer ier. If ier= 0, then no errors were encountered, convergence was 
achieved, and the final values of the parameters of the process are returned in 
alpha, beta, and rvar. If ier= 1, then no errors were encountered, but con- 
vergence was not achieved. The values of the parameters when the algorithm 
stopped are returned. Thus DTARMA can be called again using these values as 
starting values. If ier is greater than one, then an errror was encountered. If 
DTARMA succeeds in finding a maximum, then the resulting estimates for the AR 
and MA coefficients are guaranteed to have characteristic polynomials whose 
zeros are all outside the unit circle. This is because within the command, we 
have actually parametrized the model in terms of the partial autocorrelations 
corresponding to the coefficients of the AR part and the coefficients of the MA 
part, and then also transformed these in such a way that they were guaranteed 
to be between —1 and 1 (see Jones (1980)). 

The basic theoretical results for maximum likelihood estimation for time 
series are contained in the following theorem, due originally to Whittle (1951). 


Theorem 3.4.1 | PROPERTIES OF MLE’s FoR TIME SERIES 


Let X, = (X(1),...,X(n))? be a finite realization from a Gaussian 
time series that has spectral density f which depends on the parameters 0 = 
(91,...,0-)7. Let 6, be the maximum likelihood estimator of 6 based on Xy. 
Then under certain regularity conditions (see Dzhaparidze (1986)), 


Vn(6n — 8) “+ N,(0,,1-1(0)), 
where I(8) is called the information matrix of 6 and has (j,k)th element 


1 f* dlog f(w) Alog fw) 1, 


ANS hy T8050 0 


cM TES Poe 
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The regularity conditions are not easy to write down, but we note that 
there is no difficulty as long as we are considering cases where f is strictly 
positive. Thus for ARMA processes we will assume that X is invertible and 
therefore that f is positive. 


Asymptotic Distribution of ARMA MLE’s 


The next theorem applies Theorem 3.4.1 to ARMA processes. 


Theorem 3.4.2 | ASYMPTOTIC DISTRIBUTION OF ARMA MLE’s 


Suppose X is an invertible Gaussian ARMA(p,q,@,8,07) process. Let 
Ry and Rz be the autocovariance functions of an AR(p,a,1) and AR(q, B,1) 
process, respectively. Let 8 = (a? ,B7,o7)?. Then 


a) The information matrix of @ is given by 


lee yeh og, 
1(8) = IPP 0, 
Symmetric : 

_ 204 


where for j,k =1,...,p andl,m=1,...,q, we have 
se. = Ry(|5 — Fl) 


1fh = Rz(\l — m)) 
te = 4h Mme 
jl 0 gle) h(e—262) 3 


b) If g = 0, then I-!(a@) = S(a@), where S(q@) is the Schur matrix corre- 
sponding to @, and thus 


Ja a-a seat 7 S(a) 0, 
n — No+1 +1) ° 
o7 — ¢7 ‘ - OF 2% 


P 


c) If p = 0, then I-1(f) = S(B), where S(f) is the Schur matrix corre- 
sponding to B, and thus 


p-B s(B) 0 
(po) ee (os | of #1) 


q 
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Proof: If X ~ ARMA(p, q, a, 8, a”), we have 
log(f(w)) = logo? + log (9(e2"™) g(e~2*#)) 7 log(h(e?") h(e~2***)) 

= logo” + log g(e?") + log g(e~?™) — log h(e?™*“) — log h(e~27#), 
which gives 


Glog fw) _ 1 


Oo? a 
d log fw) sais 1 2nijw 1 —2nijw _ 1 

da; g(e2miw)° F glen 2miwy © a Ie) 2% (w), 

where A 
; (w) =. eritg (enti) a em and g (etre): 
Similarly, 
A log f(w) 1 
0B; Tne vile 

where 


v;(w) = e2ntde jy (ernie) ue Soph aa Calta B 


From this we have, for example, 


log f(w) Alog f(w) _ 1 
do ee dope = Ip(e2miny 94 (+) Ox () 


e27i(j+k)w - e~27i(j+k)w e27i(j—k)w e7 27i(j—k)w 


77 eo e-2e) a gle) att lg(e2"*~)|2 th lo(e2**)|2 ‘ 


The integrals of the last two terms in this sum are Ry (|j — k|) by definition, 
while the integrals of the first two terms in the sum are zero since 1 /g(z) can 
be expressed as a power series in only nonnegative powers of z (since the zeros 
of g are all outside the unit circle): 


1 co 
k 
9(z) » 
say, and thus, taking the first term for example, 


1 G2mi(j-+k)w ea alors 
3 ni(j w Tt m)w 
i galesni) du) = i e a oy WYme du) 


l=0 m=0 


ee Se 1 aa 
= SY arm [eros as, 
0 


!=0 m=0 
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and this integral is always zero since 7 +k+/+m > 0. Thus we have established 
the expression for I4*. The other expressions in part (a) are established in the 
same way, while parts (b) and (c) follow easily from part (a). 


The COEFFCSD Command 


The COEFFCSD command finds the standard errors for the estimates of 
the coefficients of an ARMA process, that is, the square roots of the diagonal 
elements of I-1(6)/n. A 95% large sample confidence interval for 6; is then 
given by adding and subtracting 1.96 times the standard error of 0; from 6;. 


3.4.2. Approximate MLE’s 


Box and Jenkins (1970) show that if X is an invertible ARMA process 
with a Gaussian error term ¢, then the likelihood function of a, 8, and o? for 
a realization X of length n can be expressed as 


L(a@, B,o?|X) = (200?)-"/?|M,, (a, B)|/? exp (- S(q, P).) 


20? 


where M,, is a matrix that is a function of @ and B, and 


n 


SCAM: 20 SG) 


t=—o0o 


with €(t) = E (e(t)|X), and the conditional expectation is calculated assuming 
that @ and # are the true values of the ARMA parameters. To find the a, B, 
and o? that maximize the likelihood, Box and Jenkins state first that the term 
|M,,(q@, B)| can be ignored, and second that the summation for S(a@, #) can be 
truncated below at some finite limit, -T, say. We describe their suggestion for 
doing the truncation. Since 


q 
e(t) = So aj X(t = J) > Bee(t —k) 


g=0 eae 
we can write the recursion 


q 


P 
é(t) = > ajE(X(t — j)|X) - 2 Pre é(t — k). 
j=0 


There are two (related) difficulties in calculating €(t) for given values of a and P. 
First, starting values for € must be obtained. Second, if 1 < t—j < n, we have 
(X(t —j)|X) = X(t—j), but for t—j < 1, we must determine E(X (t—j)|X). 
Box and Jenkins suggest a procedure that solves both problems at the same 
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time. First they note that for an invertible ARMA model, g(L) X(t) = h(L)e(t), 
E(X(t)|X) is the same as E(X(t)|X) for the “backward” model g(La)X@)= 
h(L~*)n(t) where 7 is also a WN(c?) process with n(t) independent of future 
X’s, that is, E(n(t)|X(t+ j)) = 0. This is based on what we saw in Section 2.4 
about the finite memory forward and backward prediction coefficients. 

Thus if we define X(t) = E(X(t)[X) and 4(t) = E(n(t)|X), we have 


X(t) = SO a(t +k) — > a; X(t + 9), 
k=0 51 


where #(t + j) = 0 ift+j <1 and X(t) = X(t) if1<t<n. This gives the 
following scheme for evaluating S(a, B): 


1. Let #(n—p+1) =---=9(n—p+q) = 0, and fort = n—p,n—p—l,...,1 
find 


d 


A(t) = Do a3X(t- j) — > Balt +k). 
g=0 k=1 


2. Calculate 


q P 
X(t) = >> Ait + k) — > aj X(t +3) 
k=0 gl 
for t = 0,—1,...,—T, where X(t + j) = X(t+j) ift+j > 1, A(t+k) =0 


ift-+k < 1, and t is determined so that X(t) is essentially zero for t < —T. 


3. Next take X(t) = é(t) = 0 for t < —T and run the recursion forward; that 
is, 


Pp q 
et) = Dl ajX(t-5)- SY Bee(t—k), t= -T,...,n. 
7=0 k=1 


Note that steps 2 and 3 are called back forecasting (or “backcasting”), 
and we are ensured that a truncation point —T will be reached because we are 
running a difference equation (which has zeros inside the unit circle) backward 
and the forcing terms 7 become zero when their indices become less than one. 

Once & and # are found to minimize S (a, B), the value of o? that maxi- 
mizes the likelihood (again ignoring the term |M,|‘/?) is given by 


G2 S(4, B) 


n 
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The SEASEST Command 

The command 

SEASEST(w,n,ords, coeffs,lags,it,eps,nback,rvar,ier,sds,e) 


will find the approximate MLE’s for the parameters of a data set W of length 
n that is input in the array w of length n where W follows a multiplicative 
subset ARMA model (see Section 2.5.6) of the form 


Devt (so42") (W(t) -= (oan) (33 2-2"-] é(t) 


The input array ords contains the five elements p, P,qg,Q,M, where M is 1 if 
p is to be included in the model and 0 if not, while the array lags contains 
u1,..., up followed by v1,..., vq ifeither P or Q is nonzero (an argument must 
be included for lags whether such an array is defined or not). On input, the 
array coeffs contains initial estimates for the coefficients of the model in the 
order a@, 8, B, and ¥ followed by one for the mean p. Starting values need not 
be included for terms not in the model. On output, coeffs contains the final 
estimates of the parameters. If M = 1 in the array ords, then the final element 
of coeffs is the approximate MLE of the constant 79 in the rewritten form 


Pp ig 
g(L)G(L)W(t) =m + A(L)H(L)e(t), = = | Da; ya)» 
j=0 =0 


of the model. 

The SEASEST command uses the Marquardt algorithm (see Box and Jenk- 
ins (1970), p. 504) which is an adaptation of a standard nonlinear regression 
algorithm using numerical derivatives. The input integer it is the number of 
iterations to allow in the procedure. Usually it=20 is sufficient. Note that 
if it is negative, then the estimates for each iteration are displayed. If it=1, 
then only one iteration is performed; that is, the output arguments sds, rvar, 
and e are formed while coeffs is unchanged. This allows the user to essentially 
evaluate the sum of squares function for several values in the neighborhood of 
coefficients judged to be the maximizing ones. 

The input real scalar eps is a convergence criterion. If the largest difference 
of successive iterates is less than eps, the algorithm is judged to have converged. 
The input integer nback is how far back in the past that the user will allow the 
back forecasting to go. Thus nback is the number T in the description of the 
evaluation of S above. If nback=0, then no back forecasting is done. Usually 
nback=20 or 30 is sufficient. 

In addition to coeffs, the arguments rvar, ier, sds, and optionally e 
are output from SEASEST. The real scalar rvar is the estimate of o?. As is 
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usually the case with regression-type procedures, we do not use the divisor 
n in finding rvar, but rather n minus the number of estimated coefficients. 
The array sds contains estimates of the standard errors of the corresponding 
elements of the coeffs array. Thus approximate 95% confidence intervals 
for the coefficients can be obtained by coeffs-sds2 and coeffstsds2 where 
sds2=1.96*sds. The output integer ier is an error indicator. If ier=0, then 
convergence was reached, while ier=1 means that one of the matrices in the 
iterative procedure was judged to be singular. Values of ier of 3 or 4 mean that 
the diagonal adjustments in the system of equations in the algorithm become 
too large or too small respectively, but that there is a good chance that the 
outputted coefficients are satisfactory. If this happens, calling SEASEST with 
coeffs slightly different than the output values can usually get it to terminate 
properly. The optional output argument e contains one step ahead prediction 
errors corresponding to W(1),...,W(n). 


Estimates for the Airline Data 
The following macro produces estimates of the parameters of the model 
W(t) = (1+ BL)(1+ yL})c(t) 


for the airline data, where W(t) = (1 — L)(1— L?) log( X (t)). 


Ss 

miele AIREST.MAC: macro to find approximate MLE’s for the 
cD ERG airline data. 

Ae sis 

5 PAUSE 

6 ;start 

7 READ(air,x,n) jread 

8 w=LOGE(x,n) 3; and 

9 w=DIFF(n,1,w) transform 


11 w=DIFF(nm1 ,12,w) 

12 nmi3=n-13 

L337 

L4c3 Initial Estimates: 

i 

16 beta=<1,12,13> 

leg ARMASEL (w,nm13 ,40,20,0,2,-2,.95,p,q,alpha, beta, rvar, ier) 
yf 

Uo? MLE’s: 

20 ; 

21 ords=<0,0,1,1,0> 

22 lags=<1i2> 

23 coeffs=<beta[1] ,beta[i2]> 

24 SEASEST(w,nmi3,ords,coeffs,lags,-10, .001,30,rvar, ier,sds) 
25 LIST(ier,rvar) 

26 LIST(sds) 


; 
10 nmi=n-1 ; 
3 
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Note that the ARMASEL command (see Section 3.5.3) is used to find initial 
estimates of the coefficients. This command finds estimates of subset ARMA 
models. The model for W can be written as W(t) = sae fre(t — 1), where 
By = B and B = y and so we take the coefficients of lags 1 and 12 from the 
output of ARMASEL as the starting values in SEASEST. Actually, using zeros as 
the starting values also leads to the same output values. This is often the 
case with SEASEST. Note that no constant term is included in the model as the 
differencing will eliminate it. The output from the macro is given in Table 3.4 
below. 


Table 3.4. Estimates for the Airline Data 


MA part of Select ARMA 


Wil Spee .00 .00 .00 -00 .00 .00 .00 .00 -00 

11 | .00 -.47 .14 
Iterations 

1 -.3697 -.4705 

2 -.3950 -.6184 

3 -.3904 -.6132 
Coefficients From SEASEST 

st | - .390353 -.613179 
ier=0 
rvar=.001363 
Standard Errors From SEASEST 

tl 083061 073945 


3.4.3. Method of Moments Estimators 


In Section 2.6 we described methods for finding the coefficients and error 
variance for AR, MA, and ARMA processes given their autocorrelations. These 
procedures are implemented in the CORRAR, CORRMA, and CORRARMA commands, 
respectively. A natural way to estimate the coefficients of these processes is 
to use estimates of the autocorrelations in these procedures. Such estimators 
are called method of moments estimators. For AR processes, the method of 
moments estimates use the Yule- Walker equations with sample autocorrelations 
substituted for true autocorrelations. The resulting estimates are thus called 
the Yule-Walker estimates and we will discuss them in detail in Section 3.4.4. 
For now, we just point out that for an AR(p) process the Yule-Walker estimates 
have the same asymptotic properties as the maximum likelihood estimators;, 
that is, they are asymptotically efficient. This is not true for the MA and 
ARMA case (see Priestley (1981), p. 355, for example). 
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MA Processes 


As was pointed out in Section 2.6, the methods for MA and ARMA pro- 
cesses involve factoring an autocovariance generating function. When we use 
sample autocovariances in the procedures, we are not guaranteed that the fac- 
torization is feasible. For an MA process, the factorization is feasible if the 
estimate 


q 
fw)= » R(v) cos 2rvw 
is positive for all w € [0,1]. 
3.4.4. Estimation for AR Processes 


Autoregressive processes are widely used in both spectral estimation (see 
Section 3.8) and forecasting. In this section we consider some non-MLE esti- 
mation procedures for such processes. Throughout the section we assume that 
the sample mean has been removed from the data being analyzed. 


Yule-Walker Estimators 


Perhaps the most natural method of estimating the parameters @ and o? 
of an AR(p, a@, 7”) process is to substitute the sample autocovariances R for the 
true autocovariances R in the Yule-Walker equations (see part (c) of Theorem 
2.5.3), and then solve the resulting sample Yule-Walker equations: 


P 
T,@ = -? and ¢?= >> 4; R(3), 


Z==0 


to obtain what are called the Yule-Walker estimators (YWE) & and 6?. Note 
that this method is known as the “autocorrelation method” in the engineering 
literature (see Makhoul (1975)). The availability of Levinson’s algorithm makes 
the YWE’s attractive from a computational point of view. Unfortunately there 
is wide evidence that the YWE’s perform poorly in some cases (see Example 
3.12, Tjgstheim and Paulsen (1983), and Newton and Pagano (1983b) for more 
information about this problem). We will see that they are ordinary least 
squares estimators for the regression model 


X(t) = —a,X(t—1)—---—a, X(t —p), t=T,...,n +p, 


where the unobserved X’s (the ones having indices 1— p,..-,Oandn+1,...,n+ 
p) are replaced by their expectation which is zero. Thus the YWE’s multiply 
the infinitely long realization {X(t),t € Z} by the “boxcar” function that is 
one for 1 < t < n and zero elsewhere. We will see that this leads to some 
very simple calculation procedures. However, it is clear that if zeros are poor 
proxies for the unobserved X’s, the resulting estimators can be poor. 
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To derive the calculation procedures we introduce the circular shift oper- 
ator. 


Definition. Let x be an m-dimensional vector. Then the vector circular shift 
operator L applied to x results in the m-dimensional vector z given by 


x(n) 
z(1) 


z=Lx= 
z(n—1) 


The operator applied to a matrix W results in a matrix U whose columns are 
the result of applying L to each of the columns of W. 


Theorem 3.4.3. | PROPERTIES OF YWE’s 


Let & and G? be the YWE’s calculated from a sample X(1),...,X(n). 
Then 


a) & and G? are the ordinary least squares estimators for the regression 
problem y = —Xa +e where E(e) = 0,, Var(e) = o7I,, y is the (n + p)- 
dimensional vector y = (X(1),..., X(n),0,...,0)", and X is the (n + p) x p 
matrix X = (x1,...,Xp), where x; = L’y. 


6p) The zeros of g(z) = Deiat &;2/ are guaranteed to be outside the unit 
circle. 


c) If X(1),...,X(n) is a sample from a Gaussian AR(p,a,o7) process, 
then 


a—a . S(a) 0, 
vn{ —> Np+i | O41, 1 : 
o> of ie pice 
Pg 


where S(q) is the Schur matrix corresponding to a. 


Proof: The inclusion of the p zeros at the end of y makes it so that 
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and thus the normal equations X7Xé@ = —XTy become T,4 = —Fp. Further, 
the usual sum of squares of residuals becomes 


P 
RSS = y'y +47 X7y = nR(0) +n) a;R(j) = ne, 
ga 


that is, ¢? = RSS/n. ; 
Part (b) follows from the positive definiteness of R and from part (b) of 
Theorem 2.6.6. 


Implications: This theorem gives two important results in addition to ver- 
ifying our observation about the use of zeros as proxies for unobserved data. 
First, the fitted process is guaranteed to be stable. Thus the spectral density 
is sure to be positive. Further, if we use the & in the AR prediction formula, 
we have 


Pp 
X(n+j)=-) > a;X(n+j-1), j=1,2,..., 
yal 


and the predictors will converge to zero, rather than diverging or oscillating 
between —oo and oo (see Theorem 1.6.1). The second result is that the YWE’s 
are asymptotically efficient; that is, they have the same large sample properties 
as the maximum likelihood estimators. 


Before turning to alternatives to the YWE’s, we discuss the simple algo- 
rithms that exist for finding the YWE’s. We do this because their derivation 
helps to motivate the alternatives. We first present a theorem (due to Cybenko 
(1983)) from linear algebra that will make the derivation simple. By the nota- 
tion y|X > (B,e,S ), we mean that the results of the regression of the vector y 
on the matrix X are the least squares coefficients 8 = (X?X)-1XTy, residuals 


e = y — XB, and sum of squares of residuals S = e” e. 


Theorem 3.4.4 | CYBENKO’s THEOREM 


Suppose X is an (n x k) matrix of full rank k such that 
y|X > (a,e,S) and z|X > (8,f,T7) 


and the matrix (y, X,z) is of full rank k + 2. Then 


a) y|(X,z) > | ,e—bf,5— er) 
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b) z|(y,X) > (er ,f —ce,T — es), 


where 


This theorem is important because of the special form of X in the re- 
gression model solved by the YWE’s. Let &1,...,@m and 6?,..., or be the 
YWE’s of orders 1,...,m given data X(1),...,X(n). Then it is clear that 4; 
and GF are the solution of the regression y,,|— Xj, where yy is the (n + m)- 
dimensional vector ym = (X(1),...,X(n),0,...,0)7 and Xj consists of the 
first 7 columns of the (n+ m) x m matrix Xm = (Lym,..., Lym). Thus the 
regression problem for order j + 1 is the same as that for order 7 except that 
one more column (namely L/+1y,,) is added. 

Let 61,...,9m represent the negative of the last elements of OF Om: 
Then in analogy with the partial autocorrelation of a time series, we call 6; the 
sample partial autocorrelation coefficient of lag 7. The following theorem gives 
a very simple algorithm for calculating the 6; ’s. The algorithm is essentially 
Levinson’s algorithm except that it accepts as input the data y rather than the 
autocovariances R. It provides a method for doing the regression calculations 
by triangularizing X rather than X7X (see Stewart (1973) for a description of 
the benefits in general of operating on X rather than X7X). This algorithm 
is referred to as a lattice filter algorithm in the engineering literature. 


Theorem 3.4.5 | A “GIVEN DATA” LEVINSON ALGORITHM 


Let ym = (y",02,)7 where y? = (X(1),...,X(n)), and let 
KS Lye Seals ayers 
Let e9 = ym, fo = x1, and Sp = nR(0) = y) View bet 
Niu ss (Xr cee eo de fea le ee ittin 
and 
Ym|Xp => (Gp, ep, Sp), Dien 
Xpqa Kas (Barter ty): Dine. TT — 1, 


Then 
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a) BSS Pia, herp Syed T 


b) The coefficients satisfy the recursion 


a, jos Ml 
a, = @p_1 + OpPpGp_1 
’ Pp = 2; »Mm, 
ap 
where Pa 
p-1©p-1 
ap = —-— D= Moose Site 
fp-1fp-1 


c) The sums of squares of residuals satisfy 


Sp = Tp = Sp-1(1—a?), Di Lenni 


d) The forward and backward residuals satisfy 
Ep = €p-1 + Apf,_1, DS coal 


f, = L(fp-1 + apep-1), Di ere gle 


Proof: The theorem is clearly true for p = 1. For p = 2,...,m, we seek the 
results of the regression y,,|(—x,,...,—x,). Making the identification y =7;,, 
% = —Xp, and X = (—x),...,—xp_1), and noting that 


—Xp|Xp—1 => (—Bp-1, —fp-1,Tp-1), 
we have from the first part of Cybenko’s Theorem that 
x Qp—1 — b(—By-1) 
a, = ; 
b 
where b = = EAN | LO Bae = ap, that is, 
js Ga + aA, 
ad, = : 
ap 


Thus to show part (b) we need only show part (a). For p = 2,...,m—1, we 
have 


By = + (XP Xs) XP xpi = (Xp Xp) PpXP Ym 
= Py(XpXp)= PpPyX; yet 


am P(X; Xp) XP ¥m = Ppdp, 
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since P? =1,,.P,Ag Ps A+ if A is Toeplitz, and 
X7 xp41 = —n(R(p),..., R(1))7 = —nP,(R(1),...,.R(p))” = PpXp ym. 


To show part (c), note that y2 ym = bret and Xam = Pein s and 
so 


Sp-1 = yy — G3_1X5-1¥m 


ated AT T 
= Xp Xp — Bp_1Pp-1Pp-1Xp_1%p 


A Wy 
= Xp Xp a Bp_-1Xp-1%p 
=tp-1) 
which means again by Cybenko’s Theorem that 


Ss = Spat —_ Gl es = Sp-i(1 — a>) = T). 


Part (a) of Cybenko’s Theorem gives the expression for ep, in part (d). 
To verify the expression for f,; we use its second part. Define the matrix 
Wp, = (—ym, Xp-1) and note that L(W,) = Xp. Thus 


fp = Xp+1 — Xpbp 
= L(xp) - L(W,) Bp 
= L(xp — Wp8,). 


Now the coefficients of the regression xp4i|Xp (which are f,) are also 
the coefficients for the regression xp|Wp since W7 W, = nl, = X?X,, and 
W? xp = n(R(p),...,R(1))? = Ke xp Thus xp — W,f, are the residuals 
of regressing x, on Wy, and making the identification z = xp, y = —Ym, 
and X = X,-; in the second part of Cybenko’s Theorem gives x» — Wy B, = 
fp-1 + dpep-1.- 


The PARCORR Command 


_ Theorem 3.4.5 shows that the sample partial autocorrelation coefficient 
9,41 is the sample correlation coefficient for the arrays e, and f, since the mean 
of each of these arrays is zero (they are residuals from regression problems where 
the observation vectors ym and Xp41 = L?tly,, have mean zero since we have 
assumed that the mean has been subtracted from X(1),...,X(n)) and ere, = 
i f,. But the jth elements of the arrays e, and f, are the residuals of regressing 
X(j) on —X(j—1),...,—X (j—p) and X(j—p-1) on —X(j—p),...,—X(j-1), 
respectively. Thus ep and f, are one step ahead, order p, forward and backward 
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sample prediction errors, respectively. This is in keeping with the definition of 
the theoretical partial autocorrelation of lag p. 

Recalling that 6; = —a;(j) and &? = S;/n, we have e9 = ym, fy = L(ym), 
and 62 = R(0), and for j =1,...,m: 


3 ff 1f}-1 
3 = @}_1(1— 83) 


f; = L(f;-1 — 6;e;-1). 


Given the partials, one can easily obtain the estimated coefficients for an AR(p) 
model by using the PARTAR command. 


Properties of Sample Partial Autocorrelations 


The sample partial autocorrelations play an important role in determining 
models for time series. In the next theorem we describe some basic sampling 
properties that they have. 


Theorem 3.4.6 | PROPERTIES OF PARTIAL AUTOCORRELATIONS 


If X is a Gaussian AR(p) process, then the sample partial autocorrelations 
for lags p+ 1 and higher are asymptotically independent and identically nor- 
mally distributed, are asymptotically unbiased, and have asymptotic variance 
1/n. 


This theorem suggests that a natural method for choosing the order of an 
autoregressive process is that order for which all of the partials of higher lag are 
outside the interval +2/,/n. Several authors (see Box and Jenkins (1970), p. 
178, for example) state that the sample partial autocorrelations for a general 
time series have sampling properties similar to those of the ordinary sample 
autocorrelations. 


The Burg and Related Algorithms 


We have now seen that the estimator by 44 can be interpreted as a sample 
correlation coefficient, a sample partial autocorrelation coefficient, or the re- 
gression coefficient for regressing e, on fp. Thus a whole family of possible AR 
coefficient estimators can be obtained by constructing alternative correlation 


242 STATISTICAL INFERENCE CH. 3: 


coefficients or regression coefficients from ep and f,, and then using Levinson’s 
algorithm to obtain the corresponding coefficients. The best known example 
of this is a method due to Burg (1967) (see also Ulrych and Bishop (1975)). It 
differs from the YWE’s in two ways. First, the inner products are not calcu- 
lated over the whole length of e, and f,, but rather only over their (p + 1)st 
through nth elements. This appears to avoid the problem of assigning zeros 
to the unobserved data values. Second, the Burg procedure uses a different 
method of estimating a correlation coefficient, namely 


where f, and é, denote vectors consisting of only the (p + 1)st through nth 
elements of f, and e,. Note that if we remove the tildes from fp and €, we 


would get bp 41 back again since eS i ef ep. 


The formula for 6,4; can be motivated from both a correlation and re- 
gression point of view. First, 0,41 is the value of a that minimizes 


S(a) = llf, me aé,||? + |lép — af, ||? 


(see Problem T3.13). Second, from a correlation point of view, Stuart (1955) 
has shown that for a random sample from a bivariate normal distribution with 
zero means and equal but unknown variances, 


n 
» ait 
i=1 
n n 
1 [oate dow 
4=1 sl 


p= 


is an asymptotically efficient estimator of the correlation coefficient of X and 
Y. Note that the true forward and backward prediction variances are indeed 
equal while the prediction errors have mean zero. 


There are other algorithms that follow the strategy of obtaining a partial 
autocorrelation coefficient and then using Levinson’s algorithm to determine 
the corresponding AR coeffficients (see Makhoul (1977)). Note that as long 
as the partials used are less than one in absolute value, then the resulting 
polynomial g(z) is guaranteed to have all of its zeros outside the unit circle. 
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Least Squares Estimates for AR Coefficients 


In the previous section we saw how the Yule-Walker estimators for an 
AR(p) are the least squares estimators for the regression of X(t) on the previous 
p X’s for t = 1,...,n +p, with unobserved X’s replaced by zeros. Another 
natural way to estimate the parameters is to do the regression only for t = p+ 
1,...,n, thus eliminating the need for replacing any unobserved variables. This 
leads to what are often referred to as least squares estimators of the coefficients 
and error variance. In the engineering literature this method is known as 
the “autocovariance method.” It is often stated that there is no Levinson- 
type recursive algorithm for finding least squares estimators for successively 
higher orders. However such a recursion has been developed in the signal 
processing literature (see DeMeersman (1975)), and Ensor (1987) has discussed 
the algorithm from a statistical point of view. However, the least squares 
estimators are still rarely used because they are not guaranteed to lead to 
positive spectra or stable predictors. 

The following macro will find least squares estimators for an AR(p). 


Depths 

2 5s LSAR.MAC: macro to find least squares estimates for 
BIS an AR(p) process given data x(1),...,x(n). 
is 

Diente INPUT: p,n,x 

Lamers 

(he OUTPUT: alpha,rvar (rvar is RSS/n) 

Cy Br 

9 PAUSE 

10 ;start 

11 nmp=n-p 

12 nmpmi=nmp-1 

13 ppi=p+1 

14 y=EXTRACT(x,pp1,n) ;observation vector is x(pti),...,x(n) 
15 xx=<0> ;we will accumulate design matrix in xx 
16 i=1 3;we’ll do it backward 

1h eee ;when i is 1 we’ll do last column 

18 ;startloop 

LS 

20 nl=nmpmiti 

21 x1=EXTRACT(x,i,nl) 

22 xx=<x1i,xx> ;put this column before the others 
23 IF(i.eq.p,endloop) ;finished? 

24 i=it+ti ;no 

25 GOTO(startloop) 

26,5 

27 ;endloop ;yes 

235 

29 xx=-xx 


w 
o 


REG(y,xx,nmp,p,alpha,rss,t,res) ;do the regression 
rvar=rss/n 


w 
ray 


214 STATISTICAL INFERENCE CH. 3. 


32 LIST(alpha) 


33 LIST(rvar) 
3.4.5. Other ARMA Estimators 


There are a wide variety of other estimation procedures that have been 
proposed for ARMA processes. Many of these consist of expressing the model 
as a regression model wherein X(t) is regressed on the previous p X’s and 
e’s at the time points t —1,...,t — q. Since we don’t know the e’s, they are 
approximated by the residuals of a high order autoregressive model fit to the 
X’s. The best known example of this idea is due to Durbin (1959, 1960). The 
stepwise ARMA procedure described in Section 3.5 below also uses this idea. 


Other ARMA estimation procedures include those proposed by Hannan 
(1970), Parzen (1971), Anderson (1975), and Tsay and Tiao (1984). 


3.5. Identifying ARMA Models 


The problem of determining what type of ARMA model best fits a data set 
has become very important in time series analysis. In this section we describe a 
variety of diagnostics that have proven useful in determining models and orders 
of models. We will then describe a TIMESLAB macro that will help guide a 
user through parts of the model selection problem. 


3.5.1. Some Useful Diagnostics 


Suppose that we have a realization X(1),..., X(n) from a covariance sta- 
tionary time series X. We saw in Chapter 2 that the true autocorrelations, 
partial autocorrelations, and spectral density have certain characteristics for 
different model types. The statements below are of necessity rather general 
and somewhat vague. In later parts of this section we will describe some meth- 
ods that have been developed to try to use these statements in a somewhat 
automatic way. 


AR Processes 


The autocorrelation function decays to zero and then oscillates about zero, 
while the partial autocorrelation becomes identically zero for lags greater than 
the true order p. The decay of p can follow a sinusoidal pattern if some of the 
zeros of its characteristic polynomial are complex. The spectral density of an 
AR process can contain very sharp peaks, while the troughs appear somewhat 
less sharp. 
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MA Processes 


The autocorrelation function is identically zero for lags greater than the 
order of the process, while the partial autocorrelation function does not become 
identically zero. The peaks in the spectral density are smooth relative to what 
can be attained for AR models, while the troughs can be rather sharp. 


ARMA Process 


Here the autocorrelation function decays in the same way as that of an AR 
process but only after lag g. Again the partial autocorrelation does not become 
zero. The spectral density function can now have either sharp or rounded peaks 
and troughs. 


The statements made above are for models that have nonzero coefficients. 
If only a few of the coefficients of a model are nonzero, then further statements 
can be made. We consider two examples of this, although there are a wide 
variety of possibilities. First, if we have an MA or ARMA model with only 
a few nonzero coefficients, then p(v) can be small for some lags smaller than 
q (see the sales data example in Section 3.6). Second, if X is an AR(p) with 
only a» nonzero, then the spectral density of X will have a very sharp peak at 
frequency 1/p (see Problem C3.20). We will consider below automatic methods 
for determining if a subset model will adequately represent a time series. 


Other Diagnostics 


In addition to the sample autocorrelations #, sample partial autocorrela- 
tions 6, and sample spectral density f, we will consider several other quantities. 


The Biased and Unbiased Residual Variances 


The “biased residual variances” oF and “unbiased residual variances” oF 
are defined by 65 = R(0) and 


’ i “ RSS(j) 

P= 21) = BSS 

=? smite AS 

i oie 
where RSS(j) is the sum of squares of residuals for the jth-order Yule-Walker 
regression model. These quantities are calculated for j = 1,...,M for some 


integer M which is chosen to be larger than any lag for which there is useful 
information in the sample correlation function. One rule of thumb that we 
have found useful for selecting M is as the smaller of n/4 and 30. 

The biased residual variances are so called because they correspond to 
using the divisor n in the regression model instead of the usual divisor n — J 
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that is used to produce an unbiased estimate of error variance in regression 
models. The unbiased residual variance is in fact the residual sum of squares 
divided by n — j. In fact, in small samples, neither of these estimators is 
unbiased, while asymptotically they are both unbiased. Note that since the 
6; ’s are between —1 and 1, the biased residual variances are a monotonically 
nonincreasing sequence, while the unbiased residual variances can in fact attain 
a relative minimum since they are the product of the decreasing sequence oF 
and the increasing sequence n/(n — j). 


It is sometimes useful to standardize the biased and unbiased residual 
variances by dividing them by R(0). This converts the biased residual vari- 
ances to numbers between 0 and 1 and essentially produces quantities that are 
analogous to 1 — R? in regression. 


Quantities for the Dual Process 


If X has a spectral density that is strictly positive, then we saw in Section 
2.5.8 that its reciprocal fi exists and is the spectral density of some process 
Y. We will call Y the process that is dual to X. Given X(1),...,X(n), 
we can produce quantities for the dual process analogous to those we have 
defined so far by estimating the inverse autocovariance function Ri, that. is, 
the autocovariance function of Y, by 


yamaley: : cos 27 Uw; 
REAP en ny om 


for v=0,...,M, and then fitting autoregressive models of successively higher 
orders and finding what are called the inverse partials and the inverse biased 
residual variances and inverse unbiased residual variances. If it is felt that X 
is either an AR process or an invertible MA or ARMA process, then we can 
also estimate Ri(v) by 


In fact, one indication that X might not be an invertible process is that Ri and 
Ri do not agree very well. We saw in Section 2.5.8 that if X is an AR(p) or 
MA(q) process, then the inverse of its spectral density is the spectral density 
of an MA(p) or AR(q) process, respectively. If X is an ARMA(p, q) process, 
then Y is an ARMA(q, p) process. 
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The Generalized Partial Autocorrelations 


We saw in Theorem 2.6.4 that the true partial autocorrelation of lag v is 
the negative of the vth prediction coefficient for the one step ahead memory- 
v linear predictor. Since these prediction coefficients satisfy the Yule-Walker 
equations (with the signs reversed), we can view 0, as the negative of the 
last coefficient of an AR(v) model fit to X. For an AR(p) process, this last 
coefficient is zero for v > p. Woodward and Gray (1981) have extended this 
idea to ARMA processes. They define the generalized partial autocorrelation 
coefficient (GPAC), ;,;, to be the negative of the kth autoregressive coefficient 
when an ARMA(k,j) model is fit to X. They show that if X is in fact an 
ARMA(p, q) process, then 


Oe sD = 


Thus if one were to create a table of 6;,,; for AR orders on the left between 0 
and P and MA orders across the top between 0 and Q, then the true orders 
p and q could be identified by finding the place in the table where henceforth 
the values across the row are constant and the values down the column are 
all zero. The following macro will form the GPAC array for a user-specified 
correlation sequence (either population or sample values) and maximum AR 
and MA orders. Note that the GPAC values for q = 0 are the usual partial 
autocorrelations, while the values for p = 0 are not included. In the macro we 
have estimated the AR coefficients for each of the models involved by using the 
CORRARMA with estimated correlations. This leads to consistent estimates. One 
could also use the iterated autoregressions method of Tsay and Tiao (1984). 


33 GPAC.MAC: macro to find the GPAC array for ARMA (p,q) 

33 processes for p=1,...,pmax and q=0,...,qmax. 

sie The macro will also display the resulting table 
sis correctly as long as qmax is less than 7. The 
ot AR orders (1 through pmax) will be on the left 
a3 and the MA orders (0 through qmax) across the top. 
3; INPUT: pmax, qmax, rho(1),...,rho(pmax + qmax) 

PAUSE 

;start 

PROMPTOFF 

Ptq=pmax*{qmax+1} 

gpac=LIWE(ptq,0,0) ;reserve space for the array 
alphai=CORRAR(rho ,r0, pmax ,rvar) ;find values for q=0 
part2=ARPART(alphal , pmax, ier) 

gpac[1]}=part2[1] 


Ree 
WNHrF OH WON AA HL WW eH 


FRR eR 
NO ph 


ied 
© © 
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20 p=1 
21 q=1 
22 jj=1 
23 5 
24 ;s81 
25 ; 

26 LIST(p,q) 

27 rvar=corrarma(rho,r0,p,q,100, .001, ier, alphai ,betal1) 
28 jj=jitt 

29 gpac[jjl]=-alphat [p] 

30 IF(q.eq.qmax,s2) 

31 q=qt1 

32 GOTO(s1) 

33 ; 

34 382 

35 ; 

36 IF(p.eq.pmax,s3) 

37 p=pti 

38 jj=jjtt 

39 gpac(jj]=part2[p] 

40 q=1 

41 GOTO(s1) 

42 ; 
43 ;s3 

44 ; 

45 qmaxp1=qmax+1 

46 LABEL(gpac)=’GPAC Array’ 

47 LIST(gpac, ptq,qmaxp1 ,8f7.3) 
48 PROMPTON 


for) 


e 


We illustrate the use of the GPAC in Table 3.5 wherein we give the GPAC 
table (1) for the ARMA(3,2) process (see Woodward and Gray (1981)) 


X(t) — 1.5X(t—1) +1.21X(t— 2) — 455X(t — 3) = e(t) + .2€(t—1) + 9e(t 2), 


(2) for the sample autocorrelation function of a realization of length 200 from 
this process, and (3) for the log (base 10) of the lynx data. Note how the 
expected pattern occurs in the true GPAC and is discernible in the one for the 
simulated data. For the lynx data, it would appear that an AR(2) would be 
appropriate. In the next section we will consider the lynx data further. 


The Prediction Variance Horizon Function 


A number of the series that arise (particularly in business and economics) 
are well modeled as an ARMA process where only a few of the moving average 
coefficients are nonzero. A diagnostic that has proven useful in detecting such 
a model is what is called the prediction variance horizon function, defined as 
follows (see Parzen (1981)). We saw in Theorem 2.4.1 that the error variance 
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Table 3.5. The GPAC Arrays for the True and Sample Autocorrelations for 


an ARMA(3,2) Process and for the Log of the Lynx Data 


GPAC Array for True ARMA(3,2) Model 


p\q vy) 1 
1 | 845 .606 
2 | -.706 -.458 
Sut 414 .836 
4 | 299.683 
5 | -.304 -.434 
6 | -.145 -.646 
ce! 245 .279 
8 | 062 .848 
GPAC Array for Sample 
p\q ) 1 
f| .780 467 
2 | -.625 -.358 
3 | -351 .878 
4 | .338 491 
5 | -.167 -.605 
6 | -.223 -.305 
7h (| oLiGe 225: 
8 | 057 .235 
GPAC Array for Log 
p\q Co) 1 
1 | .785 .433 
2 | -.720 -.795 
3° 2=21437 "9872 
4 | -.206 -.283 
5 | .115 265 
6 | -085 -.196 
wil -208 .162 
8 | .118 -.060 


of the optimal infinite memory, h step ahead forecast is given by 


have “flat spots” corresponding to the zero MA coefficients. 
We define the prediction variance horizon (PVH) as 


h-1 
, o? eR=0 Fs 


——— 


PVH(h) 


2 3 4 5 
.391  .328 1.356 1.632 
-070 2.073 -.119 5.367 
455 .455 .455 .455 
-000 .254 .156 .418 
000 .000 .420 264 
-000° .000° =.000. 217 
-000 .000 .000 .000 
-000 .000 .000 .000 

(n=200) from ARMA(3,2) Model 

2 3 4 5 
PLES Pe 704" "4" 7416 1 23777 

131 -.824 2.780 -1.489 
-578 .578 .573  .490 
-000 70.991 1.979 .062 
045 .044 6.816 -.651 
.140 -.154 -.167 -.249 
-078 .463 .024 -1.598 

-.224 -.191 -.248 -.240 
(base 10) of lynx data 

2 3 4 5 
.389 3.733 1.256 .786 
845 -.872 -.890 -.973 
955°" £130°°2.813) ~ [300 
586 .353 .386 .258 
-536 -.056 -.186 -4.525 
-528 -.019 .896 -2.246 
-146 -.296 -.309 -.465 
-227  .067 1.245 -.164 


h-1 
2aee. 2D 2 
o,-—9 Bre 
k=0 


where the (’s and o? are the coefficients and error variance of the MA(oo) rep- 
resentation of X. We can estimate these quantities by first fitting a (possibly 
high order) autoregressive model to the data and then inverting the character- 
istic polynomial of the fitted process. If the data come from a process having 
a subset MA part, then we would expect that the estimates of the B’s would 


R(0) 


=A ee & 
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t Pe for Aipline Data, PUH(@)=,68367 


“te 4.@ 8.0 12.08 16.00 20.8 


Figure 3.9. Values of PVH for the First and Twelfth Difference of the Log of 
the Airline Data. 


Note that 0 < PVH(h) < 1 since we also know that 
lim 0? = R(0). 
h--00 


To illustrate the use of PVH, consider Figure 3.9 which contains the graph 
of the PVH for horizons 1 through 40 for the first and 12th differences of the log 
of the airline data (see Example 3.13 for the macro that produced the values 
of PVH). Note the jumps in the PVH for h=1 and 12. This suggests that the 
data contain a subset MA part having at least lags 1 and 12. 


3.5.2. The AIC and Related Criteria 


The problem of order determination in time series analysis is similar to 
the general statistical problem of determining which of a set of possible models 
best fits a set of data. One very important method for making such a choice is 
due to Akaike (1971) and is thus called Akaike’s Information Criterion (AIC). 
Although there are a variety of ways to describe AIC, one simple motivation 
is as follows. If we have two competing models for a data set and they have 
the same number of parameters, then we would probably choose the one that 
best “fits” the data in some sense. For example, in regression, if we have two 
possible models for the dependent variable and each is in terms of a subset of 
q independent variables, then we would probably choose that model that leads 
to the smaller sum of squares of residuals. If competing models have different 
numbers of parameters, then we would not want to choose one based merely on 
goodness of fit. Again, in regression, it is a well-known phenomenon that adding 
a variable to a model is guaranteed to decrease the sum of squares of residuals, 
but also tends to increase the variability in predictions resulting from the larger 
model. We have also seen this phenomenon in the problems of smoothing that 
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we have considered in this book (see the moving average smoother in Chapter 
1 and the problem of smoothing the periodogram in Section 3.2). In such 
problems, increasing the complexity of a model may reduce variability, while 
at the same time increasing bias. 

Using general principles of information theory, Akaike proposed measuring 
the “goodness” of a model that has r parameters 6 = (1,...,0,)? by 


~ 


AIC(r) = —2 log L(8) + 2r, 


where L is the likelihood function of 8, and 6 is the maximum likelihood esti- 
mate of 8. The first term is the measure of how well a model fits the data, while 
the second is a penalty for the number of parameters in the model. The AIC 
is then used to choose the model that has the smallest value of the criterion. 
Thus if a term in AIC is the same for all competing models, it can be deleted. 
For ARMA modeling, this leads to considering 


AIC(r) = nlog &? + 2r, 


where 6? is the maximum likelihood estimate of the error variance for the 
model having r parameters. 


Using AIC for AR Models 
AIC is very easily applied to the problem of AR order determination. For 


example, the following macro will use it to choose the order of an AR model 
for a user-specified data set. 


ib Ge 

2 3;  AIC.MAC: This macro finds aic(k) for k=1,...,m 
34a for data x(1),...,x(m), and returns the 
4) Sis chosen order p. 

5 55 

ORR INPUT: x,n,m 

Le Be : 

8 53; OUTPUT: p, aicO, aic(1),...,aic(m) 

9 33 

10 PAUSE 

11 ;start 

12 y=SUBMNS(x,n,1,xbar) ;subtract mean 


13 part=PARCORR(y,n,m,rvar) ;find sigma hats (rvar) 

14 aici=LOGE(rvar,m) 

15 aicl=n*aici 

16 aic=LINE(m,0,2) 

17 aic=aictaici 

18 LABEL(aic)=’AIC Criterion’ 

19 MAXMIW(aic,m,amax,imax,amin,p) ;find minimum value 
20 rho=CORR(x,n,0,0,1,r0,per) ;find aic(0)=n log(ro) 
21 aicO=LOGE(r0) 
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22 aicO=n*aicO 


23 IF(amin.1t.aicO,end) ;check if it’s really order 0 


24 p=0 
Pia 
26 ;end 
20 a 


28 CLEAN(y,part,aici,amax, imax,amin,r0O,xbar,rvar) 


For an AR process, the value of AIC for order 0 is just AIC(0)= n log R(0). 
If we apply this macro to the critical radio frequencies data, we obtain the 
values of AIC in Table 3.6. Note that order 17 is chosen, with the value for 
order 14 being just a little larger than that for order 17. For the log of the lynx 
data, AIC chooses order 11 (see Problem C3.14). 


Table 3.6. Values of AIC for the Critical Radio Frequencies Data 


AIC Criterion 


Wi 45.178100 -17.545470 -16.150540 -15.371550 -14.322820 
6 | -27.734450 -30.535160 -38.921380 -47.714840 -72.871150 
11 | -80.831830 -91.284290 -124.621300 -134.903300 -132.925800 
16 | -133.348200 -135.238500 -133.812100 -131.959900 -130.779100 
21 | -128.786800 -127.054800 -126.048800 -124.377200 -125.346500 
26 | -123.501000 -121.502900 -119.560900 -117.599600 -115.663100 


aic0=427 .409300 


A large amount of effort has been made in investigating the properties of 
using the AIC for AR processes. 


Theorem 3.5.1 | INCONSISTENCY OF AIC FOR AN AR(p) 


If X is a Gaussian AR(p) process and p is the order chosen by the AIC 
based on a realization of length n, where the maximum order allowed is K 
which is known to be greater than p, then 


lin) Pr(p =k) = 


n->0o 


{ie > 07 Roy 
0, k<p, 


where g(k, K) is a function only of k and K. 


This result (due to Shibata (1976)) says that the AIC is not guaranteed to 
choose the right order no matter how long a realization one has. Thus it is said 
to be inconsistent. Shibata (1976) calculated the function g(k, K) for0< p< 
k < K = 10, and found that for p = 0 through 8 the asymptotic probability 
of correctly choosing the order was between 70 and 78% (see Problem T3.14). 
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Note further that the theorem also guarantees that AIC will not asymptotically 
underestimate the order. This will be important in autoregressive spectral 
estimation (see Section 3.8). 


The Hannan-Quinn Consistent Modification of AIC 


The AIC can be easily modified to produce a consistent estimate of the AR 
order. This fact is given in the following theorem due to Hannan and Quinn 
(1979). 


Theorem 3.5.2 | ConsIsTENT AR ORDER DETERMINATION 


If the 2k in the AIC is replaced by 2kcloglogn for any constant c > 1, 
then the resulting order determining criterion will be consistent; that is, the 
probability of correctly choosing the order of the AR process converges to 1 as 
n—> oo. 


This theorem says that if we replace the penalty function 2k by one that is 
just a little larger (note that log log n goes to 00 very slowly, see Problem C3.15), 
then we no longer will tend to overestimate the order. Unfortunately, there is 
evidence that seems to indicate (see Hannan and Quinn (1979)) that in small 
and moderate sized samples, the consistent methods tend to underestimate the 
order. 


The CAT Criterion and AR Models 


Another criterion that is useful for determining the order of an autoregres- 
sive process is the CAT criterion due to Parzen (1977). Instead of assuming 
that X is an AR(p) process, Parzen considered the problem of determining the 
order p of an autoregressive process that approximates in some optimal way 
the behavior of an arbitrary covariance stationary time series. We will discuss 
this further in Section 3.8. 


Definition. Let ¢7,...,¢%, be the error variance estimates from the Yule- 
Walker equations based on a sample of size n, and let 


Then the criterion autoregressive transfer function (CAT) criterion for order k 
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is defined to be 


Note that the natural way to define CAT(0) would be as just —R(0), 
in which case it has been shown that the AIC and the CAT criterion are 
asymptotically equivalent (see Bhansali (1986)). Parzen has suggested the 
definition above so that CAT can be used as a test for white noise. As we 
pointed out above, if X is an AR(0) (that is, white noise) process, then AIC 
(and hence CAT with CAT(0)= —R(0)) would asymptotically only choose 
order zero approximately 75% of the time. In Example 3.14, we consider the 
level of significance of the modified CAT criterion as a test for white noise. 


AIC and ARMA Models 


The AIC is also applicable for choosing the order of ARMA processes. 
Ideally, we should use the exact maximum likelihood method for each possible 
order and then calculate AIC based on the maximized likelihood. Unfortu- 
nately, this is very time consuming and in some cases (particularly when the 
process is close to nonstationary or noninvertible) it is difficult to get the max- 
imization procedure to converge. An alternative is to use the estimate of the 
error variance obtained from the method of moments estimator for each order. 
This also can be difficult to implement since in some cases the factorization 
of the covariance generating function becomes infeasible (see Section 2.6.3). 
Thus in TIMESLAB, we suggest using the consistent but inefficient method 
described in Section 3.5.3 to estimate the error variance for each order (see 
the ARMAAIC macro described in Example 3.15). Hannan and Rissanen (1982) 
have considered the properties of this method of determining p and qg. Once an 
order is determined, one can use the exact (DTARMA) or approximate (SEASEST) 
likelihood methods for estimating the parameters of the process. 


3.5.3. The Stepwise ARMA Method 


Since an ARMA model looks very much like an ordinary regression model, 
it is natural to apply regression techniques in their analysis. In fact we saw in 
Section 3.4.4 that applying ordinary least squares to autoregressive data leads 
to asymptotically efficient parameter estimates. In the MA case this was not 
true. In this section we extend the regression analogy to the ARMA case by de- 
scribing the TIMESLAB command ARMASEL which performs a stepwise ARMA 
procedure. See Section A.5.4 for a general discussion of stepwise regression. 
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Suppose we feel that an ARMA model of some order (p,q) should fit a 
data set X(1),...,X(n) and (P,Q) is an upper bound on the order. Then in 
analogy with stepwise regression analysis, we would write 


X(t) = —a, X(t— 1) —---—apX(t— P) + Bye(t— 1) + --- + Bge(t—Q) + €(2), 
that is, 
y= W#+.e, 
where, for r = max(P,Q), we have 
y =(X(r+1),...,X(n))?, 0= (—a?, pT)?, e= (e(r + 1),...,€(n))7, 
and W = (X,E), where 
X(r) Pom y PAP SAP) €(r) pen e(h—()) 


X(n—1) ... X(n—P-—J) e(n = 1) 64...) e(n—Q-—1) 

If we knew the elements of E, then we could form 

XPRUXTE XTy 
=|E?X ETE Ely |, 

y7X yTE yTy 
and then use ordinary stepwise regression to explore the problem of determining 
which elements of a;,...,ap and f1,...,B@q are nonzero. There have been a 
number of suggestions for finding proxies for the e’s, most of them equivalent 
to estimating the e’s by one step ahead prediction errors for a high order AR 
process fit to the data. The ARMASEL command proceeds as follows. The 


matrix A consists of lagged inner products of X’s and/or e’s. Thus for large 
n, A should be close to 


W'w wy 
ss vi ig 
YAW. iY 


xx Txe rxx 
1 
Ar=- Tee TXe ) 
n 
Symmetric Rxx (0) 


where 
(Txx);,=Rxx(li-kl),  j,k=1,...,P 
Xcite) eet ee Ret. 
(Tee) x =Rxx(li-Fl), 5k =1,...,Q 
(rxx);=Rxx(j), f=1,...,P 


(xe); = Rxe(—J), j a ly o° Q: 
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We will estimate these auto- and cross-covariances and then apply stepwise 
regression to the matrix A, which is the matrix A, with the estimates replacing 
the true covariances. 

If X ~ ARMA(p,q, a, 8,07) is invertible, we can write X as an infinite 
order AR process and thus for some possibly large order s we should have 


s 
et) = Sry X(t-3). 
j=0 
This gives by the Filter Theorem 


Ree(v) = 9) 4 Rxx (5 +0 — k). 


j=0 k=0 


Further, for positive v, X(t) should be close to uncorrelated with «(t+ v), that 
is, Rx-(v) = 0, while for v < 0, we have 


Rxe(v) =) 4jRxx(v— 3). 
j=0 


Note that if we know only Rxx(0),...,Rxx(M), then we can only calculate 
Ree(v) and Rx.(—v) forv=0,...,M—s. 

Thus for given M and s we estimate the Rxx’s by sample autocovari- 
ances, then estimate the y’s by the Yule-Walker equations, and then find the 
remaining elements of A, using the above expressions with estimates replacing 
the true values. 


The ARMASEL Command 
The command 
ARMASEL(x,n,m,s,k1,k2,kopt,pval,p,q,alpha, beta,rvar, ier) 


carries out the procedure described above. The first eight arguments are input 
and the last six are output although alpha and beta can be used as both input 
and output if kopt is negative as described below. The final model chosen has 
orders p and q returned in the integers p and q, coefficients a and # returned 
in the arrays alpha and beta, and noise variance returned in the real scalar 
rvar. The output integer ier is 0 if no errors are encountered, while it is 1 if 
the matrix being swept is judged to be singular at any step. 

The arguments x, n, m, and s contain the data, the sample size, and M and 
s as described above. The arguments ki, k2, kopt, and pval allow the user to 
use ARMASEL in a variety of ways. First, the real scalar pval is the “p-value to 
enter” at each step of the stepwise procedure. Thus using a smaller value for 


SEC 3.5 IDENTIFYING ARMA MODELS 22H 


pval allows more variables to enter the model. Then m, s, ki, k2, and kopt 
(and possibly alpha and beta) together describe what lags are contenders for 
inclusion in the model. First, the largest AR or MA lag possible is m-s as 
described above. Then we have the following rules based on the value of kopt. 


kopt=0 - The k1 AR lags (out of the possible m-s) having the largest 
Rxx(v) in absolute value and the k2 MA lags having 
the largest Rx-(—v) in absolute value are contenders for 
inclusion. 


kopt=j>0 - This is the same as kopt=0 except that the stepwise pro- 
cedure continues until j lags are included. 


kopt=j<0O - The ki AR lags that are entered in alpha and the k2 MA 
lags that are entered in beta are forced into the model. 
Thus kit+k2= —j. 


The following code will use ARMASEL to estimate the coefficients of a full 
ARMA(p, q) model for user-specified m and s such that m-s> max(p, q): 


alpha=LINE(p,0,1) 

beta=LINE(q,0, 1) 

PPq=ptq 

kopt=-ppq 

ARMASEL(x,n,m,s »P,q,kopt,pval,p,q, alpha,beta,rvar,ier) 


These estimators can be used as starting values for one of the more efficient 
estimation procedures that we have discussed (see the SALESPRD .MAC macro), 
or to calculate the AIC (see the ARMAAIC.MAC macro). The following code can 
be used to extract from the arrays alpha and beta the information needed 
by the SEASEST command if the user wants to produce asymptotically efficient 
estimators of the coefficients in the subset ARMA model as chosen by ARMASEL: 


lags=<0> 

coeffs=<0> 
lagsb=EXTRACT(beta,q,0,ne,nlagsb) 
IF(nlagsb.eq.0,s1) 

lags=<lagsb, lags> 
coeffsb=EXTRACT(beta, lagsb,nlagsb) 
coeffs=<coeffsb,coeffs> 

is 
lagsa=EXTRACT(alpha,p,0,ne,nlagsa) 
IF(nlagsa.eq.0,s2) 
lags=<lagsa,lags> 
coeffsa=EXTRACT(alpha, lagsa,nlagsa) 
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coeffs=<coeffsa,coeffs> 
382 
ords=<0 ,nlagsa,0,nlagsb,0> 


Subset Autoregression 


Subset AR models have been found to be particularly useful in representing 
some time series. For example, Tong (1977) found that for the log of the 
lynx data, an AR(11) model having only coefficients of lags 1, 2, 4, 10, and 
11 nonzero has a value of AIC smaller than that of any full AR model (see 
Problem C3.14). The ARMASEL command can be used to fit such models. 

Instead of looking for an AR model which has some zero coefficients, one 
alternative would be to look for an AR model which has some zero partial 
autocorrelations. Thus for data X(1),...,X(n), one could calculate the partial 
autocorrelations 6,,...,6y for some M, then determine in some way a subset 
of these coefficients that are not significantly different from zero, and then use 
Levinson’s algorithm backward to find the corresponding AR coefficients (see 
the PARTAR command). 

One possibility for finding a subset of partials would be to start with no 
partials in the model, and then in a stepwise fashion add a partial, update 
the residual variance, and then find AIC for this subset. At each step the 
remaining partial having highest absolute value is the contender for addition, 
and the process stops when adding a partial would not decrease the AIC. The 
residual variance is easily updated since if we denote its value for k partials by 
62, we have from Levinson’s algorithm 


a — ef_1(1 i 63), 


where 6? is the new partial being added. 

Another possibility would be to include only partials that are larger in 
absolute value than some threshold value. One possibility for this threshold is 
2/.\/n which is the threshold for partials for lag greater than p for an AR(p) 
process. 


The DTAR Command 
The command 
alpha=DTAR(X,n,m,iopto,ioptl,ioptm,p,r0,rvar[,cat]) 
can be used to determine full and subset AR models according to a variety of 
rules. The array X and the integer n contain the data set to be analyzed and 


its length, respectively. Note that the sample mean should almost always be 
removed from X prior to calling DTAR as it does not do such a subtraction (it is 
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Table 3.7. Output of the DTAR Command for the Critical Radio Frequencies 
Data 


i part (i) brv(i) urv(i) cat(i) 
1 - 893475 - 201702 - 202546 -4.916577 
2 - .486187 - 154024 -155319 -6 .390981 
3 -050179 - 153636 -155581 6.353332 
4 .071237 - 152857 - 155448 -6 .332052 
5 ~- .062895 -152252 - 155492 -6 .303437 
6 . 249393 - 142783 -146444 -6.672332 
Us - 140727 - 139955 -144160 -6.751625 
8 . 205798 . 134027 -138649  -6.997269 
9 . 209706 - 128133 -133126 -7.265221 
10 - 327085 -114425 -119400 -8.093822 
lp! - 201627 -109773 -115046 ~-8 .374556 
12 — .224861 - 104223 - 109708 -8.759499 
13 - .370015 089954 -095105 -10.115280 
14 — .223356 . 085466 -090760 -10.572720 
15 - .009668 -085458 -091155 -10.479280 
16 -.100213 - 084600 -090643 -10.495350 
17 -.126803 - 083240 -089585 -10.579070 
18 -048859 - 083041 -089774 -10.509190 
19 - .024806 .082990 -090125 -10.419610 
20 ~ .058374 -082707 -090226 -10.360990 
nord = 17 


possible to imagine situations where one might want to perform the analyses in 
DTAR on data that have nonzero mean). The integer m contains the maximum 
AR order to allow, while the three integers iopto, ioptl, and ioptm contain 
the user’s choices about the order to use (1 means use order m while 2 means use 
an order-determining criterion), whether or not to display a table of partials 
and biased and unbiased residual variances and the CAT criterion for orders 1 
through M (1 means yes and 2 means no), and finally which algorithm to use. 
Note that ioptm can be 1, 2, 3, 4, 5, or 6. For 1, 3, and 5, the algorithm 
described in Theorem 3.4.5 is used, while for 2, 4, and 6, the Burg algorithm 
is used. Choices 1 and 2 mean to fit a full AR model of either order m or that 
chosen by CAT depending on whether iopto is 1 or 2. Choices 3 and 4 mean 
to use AIC to find an AR model corresponding to a subset of partials as found 
by AIC, while choices 5 and 6 mean to use the 2/,/n threshold to find such a 
subset of partials. In any case, the output variables Pp, alpha, and rvar contain 
the order, coefficients, and error variance of the output model. The output real 
scalar r0 contains the sample variance, and the optional output argument cat 
contains the values of the CAT criterion for orders 1 through m. To illustrate 
the use of DTAR, consider Table 3.7 where we display the output of applying it 
to the critical radio frequencies data. The CAT criterion chooses order 17 (as 
did the AIC), and the value for order 14 is just slightly higher than that for 
order 17. 
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3.5.4. A Macro for Model Identification 


We have seen in this section a wide variety of methods for choosing models 
for data. Because there are so many methods, the problem of model identifi- 
cation is very difficult. Thus we have included a macro with TIMESLAB that 
will lead a user through many of the methods that we have described. This 
macro is called ID.MAC. A listing of it is included at the end of this chapter (see 
Example 3.16). The macro introduces each part of the identification process 
with a screen describing what the user will see and what possible action should 
be taken in response to the nature of the diagnostics. Readers are urged to 
use this macro for a data set of their choice while looking at the listing of the 
macro. This will give experience both in identifying models and in how such a 
macro is written. 


3.6. Box-Jenkins Modeling and Forecasting 


The most popular ARMA-model based forecasting method is one pro- 
posed by Box and Jenkins (1970). The method uses the traditional statistical 
modeling strategy of (1) model identification, (2) parameter estimation, (3) 
diagnostic checking, and (4) consideration of alternative models, if necessary, 
within the class of multiplicative seasonal ARIMA models (see Section 2.5.7) 
to obtain a model suitable for forecasting. In this section we describe the vari- 
ous steps in the method and illustrate the TIMESLAB commands that can be 
used to produce Box-Jenkins forecasts. Note that the method uses only some 
of the procedures that we have described so far in the book. For example, most 
computer implementations of the method use none of the automatic model de- 
termination procedures. Also, it is only concerned with methods that are done 
in the time domain. The appeal of the method is its simplicity and the wide 
availability of computer software for using it. 


The Models 


The Box-Jenkins method for data X(1),..., X(n) assumes that if we trans- 
form X to a series W by 


w(t) =(1- L)4(1— 15)? ¥(), 
where 


fesse 440 
Y (t) = 


log(X(t)+m), A=0, 


and m is chosen to make X(t) + m positive for all t (unless A = 1), then a 
model of the form 


P Ez q Q 
(3: ot) (>: att) (W(t) — 4) = (s2au') (dor) €(t), 
j=0 k=0 1=0 r=0 
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with € ~WN(o”), will adequately represent W. Note that this model for W can 
be written as an ARMA(p+ PS,q+QS ) model by multiplying the polynomials 
in the model. One of the appeals of the Box-Jenkins models is that many of 
the coefficients in this general ARMA model are allowed to be zero. Thus 
this class of models allow us to represent data parsimoniously, that is, with a 
small number of parameters. This is the same idea as in the stepwise ARMA 
modeling embodied in the ARMASEL command (see Section 3.5.3). 

The class of models for W is called the multiplicative seasonal ARMA 
model with orders (p, P,q, Q,d, D, S)and coefficients (a, 8, B,). In this book 
we will generalize this model slightly by removing the restriction that the pow- 
ers of L in the second and fourth polynomials above are of the form kS and 
mS. This allows us to include the general subset ARMA model in the class of 
models that we can consider. Thus we have the model 


9(L)G(L) (W(t) — p] = A(L) A(L)e(2), 


where 


P P 
(L) =) ajL/,  G(L) = D> L"*, 
g==0 k=0 


q Q 
h(L)= DUAL,  H(L) = > mL, 
l=0 m=0 


are called the full and subset AR and full and subset MA operators, respec- 
tively. Each of these is assumed to have all of their zeros outside the unit circle, 
so that the model is stationary and invertible. We can also write the model 
with a constant term 79, that is, 


9(L)G(L)W(t) = r + A(L)H(L)e(t), 


where 


P te 
T = 9(1)G(1)u = (3: ») (>> | LL. 
j=0 k=0 


From our previous discussions, it should be clear that the Y transform is 
used to stabilize variance, the W transform removes trends and cycles, and the 
model for W represents the behavior in the data after this “preprocessing.” In 
the remainder of this section we describe and illustrate the four steps in the 
Box-Jenkins procedure. 
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Figure 3.10. Example of Identifying X. 


Model Identification 


The model identification step itself consists of three parts: 


1. Determine if the power transform from X to Y is required, and if so, what 
values of m and A should be used. Note that m is only included to make 


X(t) + m positive for all t. 


2. Determine if the differencing transform from Y to W is necessary, and if 


so, what values of d, D, and S should be used. 


3. Determine if an ARMA model is required to model W, and if so, what 
values of the orders and lags should be used. 
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Identifying \ 


This step is used either (1) to stabilize the variance of X or (2) to produce 
a time series Y for which the differencing step can eliminate any trends and/or 
cycles. This step is not needed for any series whose variance appears constant 
and which has no trends or cycles. To illustrate a case where the power trans- 
form is needed, consider the series in Figure 3.10, which consists of the sales 
of an industrial company for 77 months (see Chatfield and Prothero (1973) 
and Box and Jenkins (1973)). This data set consists mainly of a linear trend 
and an annual cycle whose amplitude is increasing with time at a faster than 
linear rate. If no power transformation is applied, no amount of differencing 
can remove the cycle. For example, we have included in Figure 3.10 a graph 
of the first and 12th difference of X. Since all of the data are positive, we let 
m = 0 and try to find a suitable value for \. 


There are a variety of possible methods for selecting A. Unfortunately, 
which method to use is somewhat subjective and depends on the nature of the 
data. The following macro tries a variety of \’s and chooses the one that has 
the highest value of R? for the regression model 


2n(t 
X(t) = a+ bt + ccos UU) 


+ €(t), 

that is, the value that leads to the series that is best fit by a linear trend plus 
a cosine curve. See Box and Cox (1964) for a general discussion of choosing 
power transformations in regression analysis. Note that the cosine is shifted 
two months because the data start in January and the maximum yearly value 
is in November. 


re LAMBDA.MAC: macro to choose the power transform for 
ore the sales data. 


are INPUT: nlam, lami, lam2 (number of powers and first 
Hs and last powers to use) 


a's OUTPUT: r2s, lam (arrays of R-Squared and lambda’s) 


;start 

READ (sales ,x,n) 

;starti 

PLOTON 

lam=LIWE(nlam,lami,lam2,1) ;lam is array of lambda’s 
=3 

npk=n+k 

cs=COS(npk,1,12) 

cs=EXTRACT (cs ,k,npk) ;shift cosine by two units 


a 
NOnwhwWN RFP OHO MONA AN Hh WwW YH 


a 
© 0 
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20 11=LIBE(n,0,1) ;form vector 1,...,n 

21 one=LINE(n,1,0) ;intercept 

22 xx=<one,1ll,cs> ;design matrix 

23 r2s=LINE(nlam,0,0) j;reserve array to get R squares 
24 i=1 ;initialize loop 

35% 

26. ;startloop 

a6 ts 


28 lambda=lam[i] 
29 if(lambda.gt..001,nolog) ;check if we need to do log 


30 y=LOGE(x,n) syes 

31 GOTO(reg) 

32 ;nolog ;no 

33 y=x" lambda 

34 ;reg 

35 yy=SUBMNS(y,n,1,ybar) ;find sum of squares of transformed data 


36 ssy=DOT(yy,yy,n) 
37 REG(y,xx,n,3,beta,rss,t,res) ;do regression 


38 sse=dot(res,res,n) ;find R squared 
39 r2s[i]=1-{sse/ssy} 
40 yhat=y-res ;Superimpose data and fitted values 


41 LABEL(y)=’lambda=@lambda@, r2=@r2s[i]@’ 
42 LABEL(yhat)=’ ” 

43 CLS 

44 PLOT2(y,yhat,n,n,2,1) 

45 IF(i.eq.nlam,endloop) ;check if loop finished 
46 i=itl ;no 

47 GOTO(startloop) 

48 ; 

49 ;endloop ;yes 

50) 

51 PLOTOFF 


In Figure 3.10 we have included a graph of R? for 51 values of A between .1 
and .6, and the graphs of X for \ = 0, A = .24 (the value giving the maximum 
R?), and \ = .6. The log transform is ruled out because of how poorly the first 
cycle of the cosine fits the data. 

The method of the previous paragraph is appropriate for these data be- 
cause the transformed data do seem to be sinusoidal. In general, one chooses 
the transformation so that the resulting data most closely match some deter- 
ministic function that can be removed by differencing. 


Identifying Differences 


In this part of the process, one looks for the least amount of differenc- 
ing required to produce a correlogram and partial correlogram that can be 
matched by that of a multiplicative subset ARMA process having only a few 
parameters. This is not always an easy process as it requires rather extensive 
experience. In Section 3.4 we described a variety of types of correlograms and 
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partial correlograms and the corresponding models. Unfortunately no such de- 
scription can be exhaustive. Note that doing more differencing leads to losing 
observations, while doing less may lead to having to use more parameters in the 
ARMA model. The probability limits for prediction that we will obtain below 
are a function of the number of observations minus the number of estimated 
parameters. Thus there is a tradeoff in degrees of freedom in these two parts 
of the identification process. 


To illustrate differencing, consider again the sales data. In Table 3.8, 
we give the values of the correlogram and partial correlogram for the power 
transformed data and for its first, 12th, and first and 12th differences. 


Table 3.8. Table of Correlations and Partial Correlations for Various Differ- 
enced Versions of the Power Transformed Sales Data 


Correlations of Y 


TP} S89 te 45" = 19"—-0f ="09°=.07 08 88 -48 .63 .69 
13 | .61 .46 .25 .03 -.14 -.21 -.19 -.08 .07 OPS. inet T fhmearse: 
25 | .38 .26 .09 -.10 -.26 -.34 -.36 -.29 -.18 =04 50070) 214 
Partials of Y 
Aol, ..89)=542 —°50 08455299243 £108 «15 4.13 -.06 .03 -.04 
13 | -.28 .04 .13 -.02 -.18 -.04 .11 -.05 -.08 pa 2) O70 O05 
25 | -.06 -.05 -.140 -.11 .02 .02 -.11 -.08 -.08 -09 -.05 -.02 
Correlations of ist Difference 
1 | .33  .40 -.07 -.29 -.57 -.49 -.54 -.25 .02 .24 -45 .61 
13 | .36 .27 -.01 -.23 -.44 -.43 -.43 -.20 -.05 ee PO eA) 
256) e290 3298 204 —. 13 = 35 =—.34 —.35 —.20 =. 10 iS »218" .39 
Partials of ist Difference 
1 | .33 .33 -.33 -.45 -.46 -.13 -.20 -.24 -.03 -.06 =,04. 7.24 
13 | -.08 -.16 .06 .19 -.03 -.14 .04 .13-.16 .01 315 05 
25 | .08 .0S .00-.08 .02 .11 .05 .04-.14 .07 301 =.01 
Correlations of 12th Difference 
1 | .40 .54 .23 .19 .07 .04 -.13 -.10 -.06 -.14 .08 =.22 
13 | -.10 -.22 -.14 -.12 -.06 -.10 -.04 -.03 -.02 -.06 -.14 -.20 
euler (i. 0S)—.091 0100.00) 06 5-10 9101) 00 =.011=.13 =. 414 
Partials of 12th Difference 
1 | .40 .45 -.10 -.12 -.02 .01 -.20-.05 .22 -.11 16 —.30 
1S gl 217 R OSs OSeiiO HOO: O1l=.08) 11103 2893 O21 
25) |55502 95522 ~.17. 0% ) 1.00.—.06) ..05/-.43.%,04 =<15 — 503 —.16 
Correlations of ist,i2th Difference 
Lel)=560 5.36 = 22 5.06 =-.08 ..11--.17 =.02 .12 -.26 .44 -.35 
13 | .18 -.14 .05 -.04 .09 -.09 .05 .00 .03 -04 -.02 -.08 
25 | -.05 .09-.11 .10-.05 .02 .04 .08-.07 .08 -.08 -.08 
Partials of ist,12th Difference 
1 | =.60 .01 .00=.10 =.13 .06 -.10 -.31 .04 Sieeeeee st 06 
13 nlin=8468=212)=216e=206%=.068:foi" ros*=y11' a4 = '65 FO2 =s11 
25 | -.29 .11 -.12 -.04 .01 -.10 09 -.08 .08 -.04 -05 -.05 
r0x=38863.330000 
r0=.397435 
r01=.083808 


r012=.036166 
r0112=.044060 
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Once the data are differenced (if necessary), the next part of the method 
is to find an ARMA model that adequately represents the differenced data. 
Here only the sample autocorrelation and partial autocorrelation functions are 
used. To illustrate this process, we continue with a discussion of the sales 
data. Note that the correlogram and partial correlogram of the first and 12th 
difference series have an obvious pattern that the others don’t. First, the lag 
one partial is large and is followed by several that are small, followed by an 
increase at lags 10 and 11. The first three correlations (—.6, .36, —.22) are 
decaying exponentially with alternating signs, indicating the presence of an 
AR(1) term in the model with coefficient approximately .6. The presence of 
large correlations at lags 10, 11, and 12 indicates the presence of a subset MA 
term in the data. Since the maximum correlation is at lag 11, it is tempting to 
conclude that the MA term should be of lag 11. The macro given below finds 
the true autocorrelation function for the best fitting models having an AR(1) 
term and then either an MA(11) term or an MA(12) term. The output of the 
macro is given in Table 3.9. 


Table 3.9. Output of the SEASEST Macro 


Coefficients From SEASEST 

| - 504522 - 780130 
rvar=.025497 
Correlations for AR(1) ,MA(11) 


1 | -.50 .25 -.13 .06 -.02 .00 .02-.06 .12-.24 .48 -.24 

13 |. 12° =<06" .03:=.02 01 00.7. 005.00) 00 4.00 5.00 00 
Coefficients From SEASEST 

| .501941 -.801871 


rvar=.024118 

Correlations for AR(1) ,MA(12) 
1] -.50 .26 =.13 .06 =-.03 .01 .01-.03 .06 -.12 .24 =.49 
13°] “524 =.12 .06 =.03 ~.02)=,01  .00' .00 (00 _.00 .00 .00 


The residual variance for the model having the MA(12) term is smaller 
than that for the model having the MA(11) term but the correlations for the 
latter model appear to match those of the data better. There seems to be no 
clear optimal choice between the two models. 


>? 

.e SALESRHO.MAC: macro to find the correlation function of 

33 the best fitting model with AR(1) ,MA(11) 

3 and AR(1),MA(12) for the ist and 12th 

“3 difference of the fourth root of the sales data. 
PAUSE 

;start 


oONanh WN FE 
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A Read and transform data: 

READ (sales ,x,n) 

;starti 

x=x7.25 

x=DIFF(n,1,x) 

nmi=n-1 

x=DIFF(nmi ,12,x) 

nmi3=n-13 

, 

;  AR(1),MA(11): 

ords=<1,0,0,1,0> 

lags=<11> 

coeffs=<0,0> 

SEASEST (x,nm13 ,ords ,coeffs ,lags ,20, .001,20,rvar,ier,sds) 
LIST (coeffs ,2) 

LIST(rvar) 

alpha=<coeffs[1]> 

beta=LINE(11,0,0) 

beta[11]=coeffs[2] 

rho=ARMACORR(alpha, beta,1,11,rvar,24,r0, ier) 
LABEL(rho)=’Correlations for AR(1i) ,MA(11)? 
LIST (rho ,24,12,12f5.2) 

' AR(1) ,MA(12): 

lags=<12> 

coeffs=<0,0> 

SEASEST(x,nm13,ords ,coeffs ,lags ,20, .001,20,rvar,ier,sds) 
LIST (coeffs ,2) 

LIST (rvar) 

alpha=<coeffs[1]> 

beta=LIWE(12,0,0) 

beta[12]=coeffs[2] 

rho=ARMACORR(alpha, beta,1,12,rvar,24,r0, ier) 
LABEL(rho)=’Correlations for AR(1) ,MA(12)? 
LIST(rho ,24,12,12f5.2) 


Estimation and Diagnostic Checking 


Once a model has been determined, the approximate likelihood estimation 


procedure is used to estimate its parameters (see the SEASEST command), while 
the portmanteau test is used to determine if the residuals of the fitted model 
are white noise (see the QTEST command). 
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Forecasting 


Given estimates of the parameters of a model that has been judged to be 
adequate, the Box-Jenkins procedure then produces forecasts as follows. For 
simplicity we suppose that yp = 0. Then we have 


g(L)G(L)(1— L)4(1 — L°)? Y(t) = A(L)H(L)e(t), 


where Y is the power transformed series. The result of using the SEASEST 
command is estimates of the coefficients of the four polynomials g,G,h, and H, 
that is, the parameters of the model for 


W(t) =(—2)7(0 = 89)? Y Gt). 


This model for W can be translated into a model for Y, and thus we can write 
Y as an ARMA model of orders p* = p+ up +d+SD and q* = q+ vq. Note 
that if differencing was done, then some of the zeros of the polynomial for the 
AR part of the model for Y will be on the unit circle. 

Now given this ARMA(p*, q*) model for Y, a recursion for the forecasts of 
Y can be used. This recursion is similar to the ones used in the approximate 
MLE procedure. Once forecasts for Y are obtained they can be converted 
to forecasts for the original series X by doing the transformation that is the 
inverse of the one done in the power transform step. 


The SEASPRED Command 
The command 


SEASPRED(X,n,ords,coeffs,lags,rvar,tf,tl,hl,conf, 
xp,xpl,xpu, ier) 


will carry out the forecast stage for any model that can be expressed in the 
general multiplicative seasonal ARMA framework that we have described in 
this book, including the Box-Jenkins models. All of the arguments of the 
command except the last four are input. The array X and the integer n contain 
the original data and sample size. Then we have 


ords =(p, P,q,Q,M,d,D,S), 


lags contains the lags for the subset AR and MA models, while coeffs contains 
the values for the full and subset AR coefficients followed by the coefficients 
of the full and subset MA coefficients, followed by the value of the constant if 
there is one, followed finally by the values of m and A for the power transform. 
The scalar rvar contains the value of the error variance. Then the integers 
tf and tl contain the range of prediction origins to use, and the integer hl 
contains the largest number of steps ahead to forecast. Thus for each value of 
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t included in the range tf to tl, SEASPRED will use X(1),...,X (t) to forecast 
X(t+1),...,X(t+hl1). These forecasts are put into the output array xp with 
the hl forecasts starting at t = tf coming first, then the ones for t = tf + 1, 
and so on. The output arrays xpl and xpu contain lower and upper probability 
limits (using the confidence level specified by the user in the real scalar conf) 
for the corresponding elements of xp. F inally, the output integer ier is 0 if no 
error was encountered or 1 if an illegal power transform is specified. 

To illustrate the use of SEASPRED, consider F igure 3.11 which contains the 
sales data with forecasts of the next 24 values of the series and 95% probability 
limits appended. The macro that produced this graph is given below. Note 
that we used the model with the MA(12) term in the macro, and that in order 
to obtain a graph with satisfactory tic mark labels on the vertical axis, we 
divided the values being plotted by 10. 


asi 

Bie ss SALESPRD.MAC: macro to produce forecasts of the next 
Ses 24 values of the sales data and to construct 
Aaa a plot of the last 36 values followed by the 
Sse forecasts and probability limits. 

City: 

aan INPUT: none 

3 5) 

9 PAUSE 

10 ;start 

11 READ(sales,x,n) 

2's 

13 7; power transform and ist and 12th differences: 

14 ; 

15 xx=x7.25 

16 xx=DIFF(n,1,xx) 

17 nmi=n-1 

18 xx=DIFF (nmi ,12,xx) 

19 nm13=n-13 

20: + 

ve wed use ARMASEL to get initial estimates: 

22 § 

23 alpha=<1> 

24 beta=<12> 

25 ARMASEL (xx ,nm13,40,20,1,1,-2,.95,p,q,alpha, beta, rvar, ier) 
26: 3 

PYigae use SEASEST to find approximate MLE’s: 

28: ; 

29 ords=<1,0,0,1,0> 

30 coeffs=<alpha[1] ,beta[12]> 

31 lags=<12> 

32 SEASEST (xx ,nm13,ords ,coeffs,lags ,20, .001,20,rvar, ier,sds) 
33 LIST(coeffs,2) 

34 ; 

35 ; use SEASPRED to get forecasts: 


w 
a 


, 
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Figure 3.11. Sales Data and Forecasts with 95% Probability Limits. 
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Sales Data and Ferecasts 


208.0 
180.00 
160.0 
10.0 
120.0 
108.00 
80.00 
60.08 
4.0 
20.08 


0.0 
1.8 24.0 48.0 72.08 0 120.08 


ords=<1,0,0,1,0,1,1,12> 

lags=<12> 

coeffs=<coeffs,0, .25> 

tf=n 

tl=n 

h1=24 

conf=.95 

SEASPRED(x,n,ords ,coeffs,lags,rvar,tf,tl,hl,conf,xp,xpl,xpu,ier) 
. The rest of the macro constructs the plot: 

5 For the vertical scale to work properly, I have to 

. divide the values by 10. 

npi=nt1 

np24=n+24 

x1=LIWE(24,n,1) 

x2=LIWE(n,0,1) 

yy=<x,xp,xpl,xpu> 

xx=<x2,x1,x1,x1> 

yy=yy/10. 

LABEL(yy)=’Sales Data and Forecasts’ 

LABEL(xx)=? ” 

nn=<n,24,24,24> 

type=<2,2,2,2> 

; Plot: I got the scale values by looking at an unscaled plot 
: and choosing values giving good tic mark labels. 


PLOTK(xx,yy,nn,4,type,0,120,0,200) 


CH. 
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3.7. Other Modeling Strategies 


An important feature of the Box-Jenkins modeling and forecasting pro- 
cedure is that the initial detrending and deseasonalizing transformations are 
done by differencing and not by regressing the data on deterministic functions 
of time. 


3.7.1. Regression with Autoregressive Errors 


Suppose we would like to model X (t) as a deterministic function of time 
and the errors appear to be autocorrelated; that is, we have 


y=Xfrte, 


where y = (X(1),...,X(n))7, X is an (n x m) matrix of fixed constants, and 
€ = (¢(1),...,€(n))” has covariance matrix ¢?V. In this section we assume 
that € can be adequately modeled as an AR(p) process for some order p. We 
can estimate the parameters by the following iterative procedure. 


1. Use ordinary least squares to find initial estimates Bo and residuals €o. 


2. Determine the order py, coefficients @, and error variance 4? of an au- 
toregressive model for eo. 


3. Create a new observation vector z, and regression matrix W, by applying 
the AR filter found in step 2 to y and X. This is done using the ARFILT 
command (see Section 2.6). 


4. Now apply ordinary least squares to z, and W, to obtain the coefficient 
estimate 8 and residuals e; = y — XA}. 


5. Return to step 2 with e; replacing eg. 
This process continues until successive iterations result in the same value 
of the AR order. The successive values of the quantities involved will have 
subscripts 1,2,.... The following macro will carry out this procedure. Note 


that it also calculates the covariance matrix of the final regression coefficient 
estimates, as well as the residuals for the final AR process. 


iy REGAR.MAC: This macro does regression with AR errors, i.e. 


i y = X beta + epsilon 


Noah OWN fF 


33 where X is n x m, and epsilon is N(0,s2*V) 


242 STATISTICAL INFERENCE CH. 3. 


LO. as INPUT: y, X, n, m 
Td ices maxp (maximum AR order) 


13 ;; OUTPUT: beta, p (AR order), resar (residuals of AR) 

ie: Sa V (covariance matrix of beta) 

15 55 

16 PAUSE 

17 ;start 

18 z=y ;we’?1l use z and W for transformed y and X 

19 W=X 

20 p=0 

Zee s 

22 ;startloop 

23. 5 

24 PROMPTOFF 

25 REG(z,W,n,m,beta,rss,t,res) ;do regression (first time is OLS) 
26 yy=MMULT(X,beta,n,m,1) ;find residuals of regression model 
27 res=y-yy 

28 rho=CORR(res,n,maxp,0,1,r0,per) ;fit AR to residuals 

29 rvar=r0 

30 alpha=CORRAR(rho,r0,maxp,n,pi,rvar,cat) ; 

31 list(p1i) 


32 IF(pi.eq.p,endloop) ;check if same order as last 
33 p=p1 

34 IF(p.eq.0,endloop) 

35 z=ARFILT(alpha,p,rvar,y,n,i,ier) ;filter y and X using new AR 


36 W=ARFILT(alpha,p,rvar,X,n,m,ier) 

37 resar=ARFILT(alpha,p,rvar,res,n,i,ier) ;resar is residuals of AR 
38 GOTO(startloop) 

aos 

40 ;endloop 

mW 

42 V=MMULT(W,W,n,m) ;now find covariance matrix of final beta 
43 V=MINV(V,m, ier) 

44 V=rvar+*V 

45 PROMPTO 


As we will see in the next section, one possible model for the log (base 10) 
of the lynx data is 


Qnt . oat 
z(t)=4p + acos 7 + bsin >= + e(t), 


where € is an AR(p). The next macro uses the REGAR macro and finds 
p = 2 with coefficients —1.037 and 0.351, while the estimates of the regression 
coefficients are 2.901, 0.354, and —0.469. 
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ee LREGAR.MAC: macro to do regression with AR errors 
35 for logiO of lynx data. 


BG y(t)= mu + a cos(2pi t/per) + b sin(2pi t/per) + eps(tx) 


omAN DAT Rh WH 


a INPUT: per, maxp (maximum AR order) 


oO 


PAUSE 

;start 

READ (lynx,x,n) 

;startl ;entry point if we already read data 
y=LOGE(x ,n) 

ee=LOGE(10) 

y=y/ee 

const=LIWE(n,1,0) 

npi=n+t1 

c1=C0S(npi ,1,per) 

c1=EXTRACT(c1,2,np1) shave to do this so ist element is cos(2pi/per) 
s1=SIN(npi,i,per) 

81=EXTRACT(s1,2,npi) 

X=<const ,c1,s1> ;X is now design matrix for regression 
m=3 

MACRO(regar ,start) ;let REGAR.MAC do the work 


NNN NNN DN KF KF RP RP BP BH Be Be eB 
Anh WNHNRrFP OH MN HAA Ph WwW w FH 


Intervention Analysis 


An important use of regression with autocorrelated errors is the situation 
where it is suspected that some outside force causes a shift in the level of a time 
series at some time point to. This force is referred to as an intervention and the 
method of analysis is called intervention analysis (see Box and Tiao (1975)). 
The basic idea of intervention analysis is that the intervention affects the mean 
of the time series in a way that can be modeled as a deterministic function of 
time, and that the correlation structure is not effected. The simplest example 
of this is the case where the mean of the series is a constant 141 before the 
intervention and a constant pz afterward, and we want to test whether [1 = po. 
If the correlation structure of the series can be adequately represented by an 
autoregressive process, then we can use the REGAR macro described above to 
estimate y; and pz by having the design matrix consist of two columns, the 
first consisting of 1’s up to and including the intervention followed by 0’s, and 
the second being 0’s up to and including the intervention followed by 1’s. Then 
we can use the results of Theorem A.5.4 to test the hypothesis that fi = po. 

Other models of the effect of an intervention are often of the form of an 
abrupt increase or decrease in the mean followed by a decay back to the original 
mean. Any effect that can be modeled as a deterministic function of time can 
be put into the framework of a regression with autocorrelated errors. 
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3.7.2. ARARMA Modeling 


One very simple method for automatically modeling and forecasting data 
has been suggested by Parzen (1982) and used successfully by Newton and 
Parzen in a forecasting competition organized by Makridakis (see Makridakis 
et al. (1984)). The method consists of two parts: 


1. For some maximum lag Mj, calculate the regression coefficients 


n—k 

>> X(t)X(t + k) 

@, = - ———_,,___ k= 11,..., Mi. 
> x?(t) 

g=1 


Then let m be the value of k having the largest |@,|, and form 


e(t) =X(t+m)+6,X(), t=1,....n—m. 


2. Fit an autoregressive process to the e’s, using maximum possible order M2 
and an order-determining criterion to determine the order p to use. In the 
general ARARMA procedure one would fit an ARMA model to the e’s, 
but in most cases an AR process is adequate. 


The first part is called the first AR since it is in essence fitting a one lag 
subset AR model to X. The second part is called the second AR. Note that the 
first AR may result in a coefficient that is greater than one, while in the second 
AR, it is recommended that the Yule-Walker or Burg estimators be used to 
guarantee that the process fit to the e’s is stationary. 

The result of this procedure is a model of the form 


(1+ BmL™)9(L)X(t) = €(t), 


where g is the AR operator determined in part 2 of the procedure. This model 
is similar in form to the Box-Jenkins model (with no MA terms) except that it 
is easily made automatic, and the data determine the nature of the first AR, 
which is analogous to differencing and is in fact sometimes referred to as “quasi- 
differencing”. If the first AR turns out to be stable, that is, the coefficient has 
absolute value less than one, then the forecasts will eventually converge to the 
mean of the observed data. As we discussed in Section 1.6.4, forecasts that 
follow a difference equation of necessity either are explosive or must converge 
to some finite value. In the short run this may not be troublesome, but if the 
analyst has a feel for the long-run nature of forecasts, then this information 
should be incorporated into whatever model is used, either by modeling this 
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behavior using a regression model, or by using differencing if polynomial growth 
is expected, or by insisting on a stable ARMA model if the series is expected 
to remain fairly constant. 


The DTFORE Command 


For a data set entered in the array x whose length is entered in the integer 
n and input integers M1, M2 (which correspond to M, and Mp2 above), and 
npreds, the command 


DTFORE(x,n,M1,M2,npreds »m,p,beta,alpha, xp) 


will fit the model described above, and will return the chosen lag m and order 
p in the integers m and p, as well as the array xp of length n+npreds which 
contains the first m+p values of x, followed by the one step ahead predictors of 
x(m+p+1) through x(n), followed finally by forecasts of the next npreds values 
of x. 


The Sales Data Again 


To illustrate the simplicity of the DTFORE command, the macro DTFORE. MAC 
given below was used to find for the fourth root of the sales data that m=12, p=2, 
beta=<-1.0566>, and alpha=<-.229,-.457>. The first AR essentially chose 
12th differences, except that the coefficient is actually more nonstationary than 
the —1 that would mean twelfth difference. In data such as this, this quasi- 
differencing will remove most of the variation in the data, and at least in the 
short term future, the model used in the second part of the model fitting has 
little impact on the nature of the forecast. Note further in this example the 
quasi-differencing will ultimately lead to more explosive growth than the actual 
12th difference. 

In Figure 3.12, we display the sales data, the values fitted by the model 
(the actual data are represented by the x ’s), and the forecasts of the next 24 
values of the series. The values to be plotted were again divided by 10 prior to 
calling the PLOT2 command to superimpose the plots of the data and forecasts. 
Thus this figure is comparable to the one using the Box-Jenkins method. 


£f DTFORE.MAC: macro to illustrate the use of the DTFORE 
35 command by finding forecasts of the next 24 
. values of the sales data. 

+3 

ty INPUT: none 

i> 

PAUSE 

;start 

READ(sales,x,n) 


OMAN DAA Rh WY 


e 
oO 
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200.0922/¢5 Data and Forecasts by DIFORE 


180.0 
160.0 
140.0 
120.8 
100.08 
80.0 
68.00 
40.00 
20.08 


0.0 
0.0 24.08 4.8 72.0 96.00 120.0 


Figure 3.12. Sales Data and Forecasts As Determined by DTFORE. 


11 x=x~.25 ;find fourth root 

12 DTFORE(x,n,12,15,24,lag,p,alphal ,alpha2,xp) 
13 LIST(lag,p) 

14 LIST(alphai) 

15 LIST(alpha2,p) 

16 x=x74. ;reconstruct original data 
17 xp=xp"4. 

18 x=x/10. 

19 xp=xp/10. 

20 LABEL(x)=’Sales Data and Forecasts’ 

21 LABEL(xp)=’ ” 

22 PLOT2(x,xp,n,101,2,1,0,120,0,200) 


If the first AR chosen by DTFORE is stable, then the SEASPRED command 
can be used to find probability limits for the forecasts. The intent of DTFORE 
however is to provide an easy-to-use command that gives forecasts that have 
been shown (in the forecasting competition) to compare very favorably with 
those given by more elaborate methods. 


3.7.3. State Space and Bayesian Forecasting 


In the recent time series literature, much attention has been devoted to 
analyzing data through the use of models that take on the structure that is used 
by the Kalman Filter Algorithm for generating forecasts (see Section A.4.3). 
Thus one models the “state” of a time series at time ¢ recursively in terms 
of the state at the previous time. If one interprets this state space model 
from a Bayesian point of view (see Harrison and Stevens (1976)), then one can 
generate what are called Bayesian forecasts. The state space formulation has 
been used extensively in a number of other contexts as well. 
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3.8. Parametric Spectral Density Estimation 


In Section 3.2 we considered nonparametric methods for estimating the 
spectral density of a covariance stationary time series. In that section we saw 
that (1) the choice of the scale parameter M is the key to finding a good es- 
timator, (2) it is difficult to produce an estimator having sharp peaks without 
introducing possibly spurious peaks as well, and (3) the fastest that the ex- 
pected mean square error can go to zero for a window estimator having lag 
window generator with characteristic coefficient q is n~29/(29+1) | no matter 
how smooth is the true spectral density (as long as f is p-differentiable for 
some p > q). In this section, we consider using the spectral density of an 
ARMA model as an approximation to that of the process being analyzed. We 
will see that this parametric spectral density estimation procedure is easy to 
implement and under a wide variety of conditions leads to estimators superior 
to window estimators. 


3.8.1. Autoregressive Spectral Estimation 


In this section we assume that the process being observed can be written 
as an infinite order autoregressive process. As we discussed in Chapter 2, this 
includes the cases of a finite order autoregressive process and an invertible MA 
or ARMA process. In the next theorem (due to Masani (1966)), we give a 
condition for a process to be expressible as an infinite order AR process. 


Theorem 3.8.1 | CONDITIONS FoR AR(oo) REPRESENTATION 


If X is a covariance stationary time series having a spectral density satis- 
fying 


0<M< fw) <<, weE(0,1], 


for some constants A, and \2, then X can be written as an infinite order AR 
process. 


All ARMA processes except those that have a zero of the moving average 
polynomial on the unit circle have spectral densities that satisfy the conditions 
of this theorem. We consider approximating the true spectral density f of 
a process that can be written as an infinite order autoregression by that of 
a pth order AR process. Given data X(1),...,X(n), autoregressive spectral 
estimation consists of three steps: (1) determining the order k of the best 
approximating AR spectral density f,, (2) finding estimates Gy(1),..., Ge (k) 


248 STATISTICAL INFERENCE CH. 3: 


and 6? of the parameters of f,, and (3) estimating f by 


a2 
és o 
fx (w) = . 2° 
k 
si Ot (j)e?™*5" 
4=0 


To determine the order, we can use any of the methods described in Section 
3.5. However, Parzen’s CAT criterion was specifically proposed in the context 
of autoregressive spectral estimation, so we describe it here. Recall that the 
CAT criterion for order k is given by 


CAT(k) = 
-(1+2) R(0), k=0, 


where G? is the unbiased residual variance for order j7. The basic motivation 
for the criterion is contained in the next theorem. 


Theorem 3.8.2 | PROPERTIES OF CAT 


Let G3 be the error variance and 


be the transfer function of the estimated AR process of order p based on a 
realization of length n from a process which can be written as an infinite order 
autoregressive process having coefficients a;,a@2,..., transfer function 


oe . 
co (Z) = dg a5 27, 
g=0 
and error variance o2,. Let CAT(p) be the CAT criterion for order p. Then 
jim E(CAT(p)) = lim Jp, 


where 
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This result shows that the CAT criterion chooses the order that results 
asymptotically in the spectral estimator that is closest (in the sense of the inte- 
grated relative mean square error measured by Jp) to the true AR(oo) transfer 
function. This is the origin of the name “criterion autoregressive transfer func- 
tion.” 

To illustrate autoregressive spectral estimation, consider Figure 3.13 which 
contains the autoregressive spectral estimators (using CAT determined orders) 
for the three series whose nonparametric spectral estimators were given in Fig- 
ure 3.8. Note how these estimators have fewer spurious “wiggles” than did the 
window estimators. This again shows that the AR method is more successful 
in resolving sharp peaks than is window estimation. A final note about the 
critical radio frequencies data is that the window estimator for truncation point 
48 did not exhibit a clear peak at the sunspot cycle frequency, while the one 
for truncation point 96 did. A similar phenomenon occurs for autoregressive 
spectral estimation, namely that the estimate for order 14 does not have the 
low frequency peak (see Problem C3.24). 

In the next theorem (due to Berk (1974)) we present the basic sampling 
properties of the autoregressive spectral estimator. 


Theorem 3.8.3 | PROPERTIES OF AR SPECTRAL ESTIMATION 


Let X be a covariance stationary time series that can be expressed as an 
infinite order autoregressive process 


da X(t i) = elt), 


where the errors € are independent and satisfy E(X*(t)) < oo. Let fp be the 
autoregressive spectral estimator of the spectral density f based on a realization 
of length n. Then 


a) If p is chosen so that 
p* < 
i)p+oo, ii) +0, and iii) Vn > |aj| +0, 
j=pt1 


as n —> oo, then for any k fixed frequencies 0 < w; <--- <u, < 0.5, the joint 
asymptotic distribution of 


Vn/p(fp(0) — £(0)), /n/p(fp(w1) — F(w1)),--- 
Vn/P(fp(we) — f(wk)), Vn/P( Fp(0-5) — F(0.5)), 
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5 gg Critical Radio Frequencies (pci?) 


1.0 0.10 0.20 0.38 0.4 0.50 


6.0 
4.8 CAT chooses order 4, Onder 17 is first 
3,68 order where sunspot cycle peak appears. 


1.0 0.18 9.28 0.30 0.48 0.50 


‘ gg hasmi tude of a Star Data (p-26) 


0.0 0.18 0.20 0.30 0.4 0.50 


Figure 3.13. Autoregressive Spectral Estimators for the Three Series Ana- 
lyzed in Figure 3.8. 
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is that of independent, zero mean, normal random variables having variances 


4f?(0),2f?(w1),...,2f?(we), 4f?(.5). 


b) If X is a finite order autoregressive process, then letting p be any 
function g of n that satisfies p ++ oo and p®/n > 0 as n - © satisfies the 
requirements of part (a), and the variance of fp(w) goes to zero at the rate 
g(n)/n asn > oo. 


c) If X is an invertible ARMA process, then p = logn satisfies the require- 


ments of part (a), and the variance of f,(w) goes to zero at the rate of logn/n 
as Nn — oo. 


Implications: This theorem shows that if we are estimating the spectral den- 
sity of a process that can be written as an infinite order AR, then the autore- 
gressive spectral estimator is consistent and asymptotically normal as long as 
the order that we use is chosen in the correct. way. Requirement (iii) for the 
choice of p shows that the choice depends on how rapidly the coefficients of the 
AR(co) go to zero. For a finite order AR process, these coefficients become zero 
past the order of the process, and we can choose p to go to infinity as slowly 
as we want. For an invertible MA or ARMA process, we could show that the 
coefficients of the AR(oo) representation go to zero at a rate bounded by p* for 
some constant p and thus choosing p = log n is sufficient (see Problem T3.15). 

The other important part of this theorem is the fact that the variance of 
the estimator goes to zero at the rate of p/n, so that if p can be chosen as a 
function of n that goes to oo very slowly (such as in parts (b) and (c)), then 
the variance goes to zero very rapidly. Again, this choice of p depends upon 
the speed at which the AR(oo) coefficients go to zero. Contrast this with the 
window case, where the rate of decrease of the variance is bounded by the 
characteristic exponent q of the window being used. This difference between 
the window estimator and the AR spectral estimator is often phrased that the 
window estimator is “saturated” (the decrease in variance is bounded by the 
nature of the window and not by the nature of the process), while the AR 
spectral estimator is not saturated. See Davis (1980) for more discussion of 
saturation. 


As an example of this difference, suppose we are estimating the spectral 
density of an invertible ARMA process both by a window estimator using one 
of the lag window generators having q = 2, and by the AR spectral estimator 
using p = logn. Then the two rates of decrease of variance are n-4/5 — n1/5 /n 
and logn/n, and the latter goes to zero faster than the former (see Problem 
C3.16). 


252 STATISTICAL INFERENCE CH. 3. 
Optimality of AIC and CAT for AR Spectral Estimation 


We saw in Theorem 3.5.1 that the AIC (and thus the CAT criterion) tends 
to overestimate the order of a finite order autoregressive process. In the context 
of autoregressive spectral estimation, it turns out that the resulting spectral 
estimator is optimal in the following sense. Suppose for a given sample size 
n, we allow the order to be chosen from the range 1 through K, where K,, is 
chosen as a function of n so that Ky, > 00 and K,/./n > 0 as n > 00. We 
can measure how well the autoregressive spectral estimator estimates a true 
spectral density f by its integrated relative squared error 


: - ea) Ww 3 
IRSE(fs) = “ (Se) du. 


Now let N = n— Ky and C,(k) = of — 0? + ko?/N, where of and o? are 
the memory k and infinite memory prediction error variances for the process 
whose spectral density is being estimated. Then for a nondegenerate (that 
is, not expressible as a finite order process) Gaussian infinite order AR pro- 
cess, Shibata (1981) has shown that the AIC is as good as any other possi- 
ble order-determining criterion in terms of having a standardized form of the 
IRSE attain an asymptotic lower bound in probability. Since AIC and CAT are 
asymptotically equivalent if we let CAT(0)= R(0), we have that CAT shares 
this optimality. 

The next theorem (due to Newton and Pagano (1984)) gives confidence 
bands for AR(p) spectra. 


Theorem 3.8.4 | CONFIDENCE BANDs FOR AR SPECTRA 


If X is a Gaussian AR(p) process, then asymptotically the probability is 
1—a that the true spectral density f lies entirely within the Scheffe-type bands 
(see part (b) of Theorem A.5.2) 


1 1 
pe StS a DY) ee IO tL, 
(0) 9 a(eayintin LoeBinlg Sf (ayte voaennols’ 
where 
: 1 P 
h,(w) = = =F7(v) +2 4(v) cos 2rvw 
p(w) Ew F(v) + Xi) 
es 
4(v) ai =a), Sli) ap (3 +), v= 0, »P 
P j=0 
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and 


x" (w) = (1,2cos2mw,... , 2. cos 27pw) 


D=BCBi 
cs S(a) 0, 

o? 2/o+ 

o74(j — 1), k=p+1, j=1,...,p+1 
Bye = 

o-? (an4j—-1 + O-741), ol apa 18 fe leet, 


S(a) is the Schur matrix corresponding to a, and in forming D estimators 
of @ and o” are substituted for the true values. For any frequency w where 
h(w) — s(w) is negative, the upper limit is taken to be oo. 


The ARSPCB Command 


For an AR(p) process whose order, coefficients, and error variance are 
entered in the integer p, the array alpha of length p, and the real scalar rvar, 
and a confidence level 1—a that is entered in the real scalar conf, the command 


ARSPCB(alpha,p,rvar,n,Q,conf,fl,fu) 


will return the lower and upper confidence bands in the arrays fl and fu at 
the q = [Q/2] + 1 frequencies of the form (j — 1)/Q. The input integer n is the 
sample size to be used in the calculations. 

To illustrate the ideas of this section, consider again the lynx data. In 
Figure 3.14 we give a plot of the AR(11) autoregressive spectral estimator for 
the log (base 10) of the data. We also superimpose the upper and lower 90% 
confidence bands (the macro that was used to produce this figure is given in 
Example 3.18). Notice that the upper limit for low frequencies has to be taken 
to be co. We use this as an indication that the data may contain a deterministic 
component. See Section 3.9 for more discussion of this issue. 


3.8.2. MA and ARMA Spectral Estimation 


The next natural step in spectral estimation is to use MA and ARMA 
models to obtain estimators. We note that MA spectral estimation (see Brock- 
well and Davis (1985)) is very similar to nonparametric spectral estimation 
in that it results in an estimator that is a finite-degree trigonometric polyno- 
mial. Thus it suffers from the same difficulty in estimating spectra that have 
sharp peaks. It does have the advantage of having the AIC available to aid in 
choosing the degree of the polynomial. 
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Confidence Bands for Lynx AR Spectra 
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Figure 3.14. AR(11) Spectral Estimator and 90% Confidence Bands for the 
Log of the Lynx Data. 


The use of ARMA models to estimate spectra does not have the weakness 
that nonparametric and moving average methods have and much attention has 
been paid recently to ARMA spectral estimation (see Gray and Woodward 
(1986) for example). However, unless the true spectral density has both sharp 
peaks and troughs, autoregressive spectral estimation, with its computational 
simplicity, should be adequate. 


3.9. Methods for Determining Periodicities 


Several of the most famous time series that have been studied in the past 
have appeared to contain cyclical components. For example, in Figure 3.15, 
we display two data sets: (1) the annual number of Canadian lynx trapped 
on the Mackenzie River for the years 1821-1934, and (2) the annual average 
value of the daily index of the number of sunspots (using the scale devised by 
Professor Rudolf Wolf in 1849) for the years 1755-1964. These data sets have 
been extensively studied over the past several years (see for example Part 4 
of the 1977 volume of the Journal of the Royal Statistical Soctety, Series A). 
The basic property of these data is that there appear to be cyclic patterns 
but that these patterns are not perfectly cyclic. The “sunspot cycle” is a well 
known phenomenon and has an important effect on radio communications. 
The cycle in the lynx data is usually explained as being due to a predator- 
prey relationship between the Canadian lynx and the snowshoe hare, its most 
important source of food. 

Time series such as the lynx and sunspot data are traditionally analyzed 
according to one of three models: 


1. As the sum of a deterministic sinusoid plus an error that is a covariance 
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209.09 Meifer Sunspot Numbers, 1749-1963 
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Figure 3.15. The Lynx and Sunspot Data. 


stationary time series, that is, 
Qrt 5 PE 
X(t)=pt acos(——) + bsin(—-) + €(t), 
Pp Pp 


where the error series € is covariance stationary, and p is the period of the 
sinusoid. 


2. As the values of a stochastic process referred to by names such as an 
‘outburst’ model (see Morris (1977)) or a filtered Poisson process (see 
Parzen (1962b)). The basic idea of this type of model is that a physical 
process builds up and then has a large surge or outburst, followed by a 
decay back to some baseline value. We will not consider this model further. 


3. As the values of a covariance stationary time series of near cyclic type. 
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For example, in Problem T2.7 we discussed the fact that the AR(2) model 
X(t) +a, X(t — 1) + a2 X(t — 2) = €(t) 


will appear roughly cyclic with period p if aj = —2cos(27/p) and az is 
close to 1. 


3.9.1. Deterministic Sinusoid Plus Error 


If we believe that a time series is actually a deterministic sinusoid observed 
with additive error and we know the period of the sinusoid, then we can use 
regression analysis to estimate the coefficients of the sinusoid and test whether 
the amplitude is in fact zero. If the errors are uncorrelated, then ordinary least 
squares can be used, while if the errors are correlated, we can use the procedure 
described in Section 3.7.1 for doing regression with autoregressive errors. In 
fact, in Section 3.7.1 we used the lynx data to illustrate the procedure. 


Estimating a Period 


If one has data X(1),...,X(n) and suspects that they contain a determin- 
istic sinusoid of some unknown frequency plus noise, then this problem is often 
referred to as a search for hidden periodicities. It seems natural to inspect the 
periodogram of the data and test whether the largest value of the periodogram 
is significantly different from zero. If not, then we would conclude that there 
are no deterministic sinusoids of any of the natural frequencies in the data. If 
the largest value is significantly different from zero, then we must ask whether 
it could happen that there is a sinusoid but it is actually at a different fre- 
quency. To do this test, we can use the results of the next theorem. In this 
theorem we assume that n is odd. The results are approximately correct if n 
is even. 


Theorem 3.9.1 | FISHER’s ExAcT TEST 


Let f (w;) be the periodogram of a realization of length n (n odd) from 
the process 


2n(t — 1) 


X(t) = acos t=") +dein +e(t), 


where the period p is a factor of n, and € is a Gaussian white noise series. Let 
m = [n/2]. Then 
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a) Ifa = b = 0, the exact distribution of 
Y His ae ACD) 
[n/2] 
> fs) 
72 
is given by 


K Vy 
Pr(g'> 2) = Deleting (“Ja —jz)™"}, z> 0, 


where K = [1/z]. 


S 


b) The chance that the test given in part ( a) will choose a frequency other 
than 1/p is less than the level of significance a that is used in the test. 


The test in part (a) is called Fisher’s (1929) exact test, while part (b) is due 
to Hartley (1949). In Example 3.17 we give a macro for calculating the p-value 
for Fisher’s test. For the log of the lynx data, we find that g = 0.5967 which 
has a p-value less than 10~*, and the maximum is at period 114/12 = 9.5. If 
one suspects the existence of r sinusoids, then the above test can be modified 
(see Problem T3.16). 

A crucial assumption in the Fisher test is that the errors are white noise. 
If they are not, that is, the spectral density of the error time series is not a 
constant, then a large value of the periodogram at a particular frequency could 
be due to either (1) the existence of a sinusoid having that frequency, (2) the 
spectral density being large at that frequency, or (3) a combination of the two. 
This is referred to as the mixed spectrum case (see Section 6.3 of Priestley 
(1981)), and unless one has some prior knowledge of the nature of f or the 
sinusoids involved, separating the two parts of the model is very difficult. 


3.9.2. Estimating Peak Frequencies in AR Spectra 


For a process of near cyclic type, the data appear to be cyclic except that 
the lengths of cycles vary from one cycle to the next. Thus such a model is often 
referred to as a disturbed periodicity model, and the analog of determining the 
period of a deterministic sinusoid is to determine the frequency w* where the 
spectral density of the process has a peak. Searching for peak frequencies is 
not difficult if the process is a finite order MA or AR process as then f (or 
its reciprocal in the AR case) is a finite-degree trigonometric polynomial and 
finding the critical values (maxima and minima) of such a polynomial is not 
difficult. Thus suppose that we have an AR(p) process. This means that 


2 
2 P 


o Tijw 
{CES Poa 
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which is the spectral density function of an MA(p) process having coefficients 
q@ and error variance 1. Note that the critical values for h are the same as those 
of f since h is the reciprocal of f. Thus peaks in f correspond to troughs in 
h and troughs in f correspond to peaks in h. Because h is an MA spectral 


density, we have 
P 


h(w) = 7(0) +2 >s ¥(v) cos 2rvw, 
v=1 


where 
p-v 
av) = > ajajs, a eae 
j=0 


Finding where a relative maximum of f occurs corresponds to finding the 
location of the corresponding relative minimum of h. Thus we need to find 
the frequency w* where the derivative h’ is zero. Given an initial guess wo for 
such a zero, we can apply Newton’s root finding method to find w* as the limit 
of the sequence wo, wi, ..., where 


A! (w3~1) 


W; —— Wi—-1 — h”(wj—1) . 


The first and second derivatives of h are given by 


P P 
h'(w) = —4x py vy(v)sin2mvw and h'’(w) = —87? oe v?7(v) cos 27uw. 
=1 


v= 


Given a realization of length n from an AR(p) process, we can estimate 
the peak frequency w* by using the above procedure with estimates of the a’s 
replacing the true values. We will denote this estimator by w. If the order 
p is unknown, then we can estimate it and the coefficients of the estimated 
order process and again use the process described above to estimate the peak 
frequency. This estimator is denoted by #;. The properties of these estimators 
are given in the next theorem. 


Theorem 3.9.2 | ESTIMATING A PEAK FREQUENCY 


Let © and Ww, be the autoregressive spectral estimates described above of a 
peak frequency w* based on a realization of size n from a Gaussian AR(p, a, 7) 
process. Then 


a) /n(@ —w*) ++ N(0,0?(w")), where 
_ b?(w*)C(a)S(a@)C? (a) b(w*) 


o? (w* 5 . 
ne (hi (w*)/27) 
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where S(q@) is the Schur matrix corresponding to a, 
b7 (w) = (sin 2mw, 2sin4mw,...,psin 2rpw), 
h(w) = o?/f(w), and C(a) is the p x p matrix having (j,k)th element 
Ci,k(@) = aj4n + on-;, 


with a, = 0 ifu>porv<0. 


b) If a consistent order-determining criterion is used for finding p, then 
the results of part (a) continue to hold for ws. 


c) If an order-determining criterion that is guaranteed asymptotically to 
not underestimate the order is used to find p, then Ws Is a consistent estimator 
of w*. 


Part (a) of this theorem is given in Newton and Pagano (1983a), while 
parts (b) and (c) are given in Ensor and Newton (1987). From these results, 
we can find a large sample confidence interval for w* as 


w + Za/27(@), 


while the lower and upper limits in a confidence interval for the reciprocal of 
w”, that is, for the period of the peak, are given by the reciprocals of the upper 
and lower limits for the frequency. Notice the important role played by the 
second derivative of h in the asymptotic variance of &. For a sharp peak, h” 
will be large and thus the confidence interval will be narrow. For a broad peak, 
the interval will be wide. We should also note the similarity of this problem 
to the problem of estimating the mode of a continuous probability distribution 
(see Parzen (1962a)). 

A crucial part of the method of this section is the fact that we have been 
finding a zero of a finite-degree polynomial. Thus if f is of the form of an 
ARMA spectral density, that is, as the ratio of two finite-degree polynomials, 
we cannot apply the above procedure. However, as long as f can be expressed 
as the spectral density of either an AR(oo) or MA(co) process, we should be 
able to apply the above procedure and obtain asymptotically good properties. 


The ARSPPEAK Command 


Given the parameters p, a, and o? of an AR process in the integer p, the 
array alpha, and the real scalar rvar, the command 


ARSPPEAK(alpha,p,rvar,n,ier,freq,se[,start]) 


will attempt to return in the real scalar freq a critical value of the correspond- 
ing spectral density, and in the real scalar se, the value of o(w) corresponding 
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Figure 3.16. Log (base 10) of the Lynx Data and the Estimated AR(11) 
Spectral Density. 


to the frequency w and the sample size that was entered in the integer n. Thus 
if ARSPPEAK is not being used for estimation, then n should be given the value 
1. The optional argument start is a real scalar input argument specifying a 
starting value for the root-finding procedure. If no starting value is included 
in the call to ARSPPEAK, then the location of the largest relative maximum of 
the spectral density evaluated at 256 frequencies between 0 and 1 will be used 
as the starting value. The output integer ier is an error indicator. A value 
of 0 means that ARSPPEAK was successful in finding a peak. Values of 1, 2, 3, 
and 4 indicate respectively that no relative maximum was found, that a zero 
second derivative was encountered, that the iterations converged to frequency 
0 or .5, and that the iterations did not converge. All of these positive values of 
ier result in no other output being formed. If f has more than one peak, then 
ARSPPEAK can be called more than once, with different starting values being 
used each time. 
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To illustrate the use of ARSPPEAK, we consider the lynx data again. While 
there is no clear agreement among analysts what AR process best models the 
log of the lynx data, we consider the result of using the order 11 process 
determined in Section 3.4. In Figure 3.16 we give plots of the log (base 10) of 
the data and the estimated AR(11) spectral density. The confidence interval 
for the period corresponding to the largest peak in the spectral density is (9.32, 
10.05) years. The macro that produced these plots is given in Example 3.19. 


3.10. Examples and Problems 


Example 3.1 | CONFIDENCE INTERVALS FOR THE MEAN 


The macro CIS.MAC given below was used to find the graphs in Figure 3.2. 


DN sis 

ae aes CIS.MAC: macro to generate nsamps realizations of length 
tees n from an AR(1i) process having coefficient alpha, 
EG and find the 95 % confidence interval for mu=0 for 
5 ea each one, both ignoring and not ignoring the effect 
ay Oe of the autocorrelation. 

© is 

8 3; INPUT: nsamps, n, alpha (real scalar) 

9 3; seed (seed for random number generator) 

LO 53% 

11 PAUSE 

12 ;start 

13 ul=LIWE(nsamps ,0,0) ;Reserve arrays to receive quantities 
14 11=LIWE(nsamps ,0,0) ;to be used to find limits. 

15 xbar=LIWE(nsamps ,0,0) ;We do it this way so that the macro 
16 rhoi=LINE(nsamps ,0,0) ;will go faster. 

17 r01=LINE(nsamps,0,0) 

18 alph=<alpha> 

19 ns=1 

20 x=wn(seed,10) ;Warm up generator 

21 PROMPTOFF ;turn off prompt 

22 

23 ;s1 

24 ; 

25 LIST(ns) ;tell the user which sample is being done 

26 x=ARDT(alph,1,1,0,n,ier,r0) ;generate realization 

27 x=SUBMNS(x,n,1,xbar[ns]) ;find mean 

28 rho=CORR(x,n,1,0,1,r01[ns] ,per) ;find rho(1i) 

29 rhoi[ns]=rho[1] 

30 IF(ms.eq.nsamps,s2) 

31 ns=ns+1 

32 GOTO(s1) 

33 ; 

34 ;s2 
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2 Form the plots: 


Lim=1 .96*{r01/n}7.5 

lowi=xbar-lim ;lowl and upi are ignoring autocorrelation 
upi=xbar+lim 

LABEL (lowi)=’ Ignoring Correlation’ 

LABEL(upi)=’ 


; Form the plot incorporating autocorrelation: 


ul={{i+rhoi}/{1-rho1}}7~.5 


lLim=limeul ;the limits now are just SQRT((i+rho)/(i-rho)) 
low2=xbar-lim ;low2 and up2 are not ignoring autocorrelation 
up2=xbar+lim 

11=<lowi ,low2> ;get max’s and min’s so plots on same scale 


n2=2*nsamps 

MAXMIN(11,n2,1max,im,lmin, im) 

11=<up1i ,up2> 

MAXMIN(11,n2,umax,im,umin, im) 

LABEL (low2)=’ Incorporating Correlation’ 
LABEL(up2)=? 

PLOT2(low1, upi ,nsamps ,nsamps ,3,3,0,n,1min,umax) 
PLOT2(low2,up2,nsamps ,nsamps ,3,3,0,n,1min,umax) 


Example 3.2. | MSE For # AND fp 


The macro MSERHO.MAC given below can be used to compare the per- 
formance of the biased and unbiased estimates of the autocorrelations of a 
user-specified AR process. A series of realizations are generated and the mean 
square error of both the biased and unbiased estimates of the autocorrelation 
function of the process is determined. The macro also keeps track of which 


estimator is closest to the true values. 


ond 
~oenmnnst Don ke WW 


12 


>; MSERHO.MAC: macro to generate a series of nsamps realizations 
23 of length n from an AR(p) process having coefficients 
33 alpha and error variance 1. For each realization, 
<5 mu=10 is added, then xbar is subtracted, and the 
33 biased and unbiased autocorrelations calculated. 
33 The average squared difference from the true 

33 correlations is calculated as well as the number 

Hs of times for each lag that the biased estimate 

es is closer. 

=z INPUT: nsamps, mn, p, alpha, seed, M (mumber of correlations) 
PAUSE 

;start 

PROMPTOFF 
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18 smp#=1 

19 x=WH(seed,10) ;Warm up random number generator 
20 rhot=ARCORR(alpha,p,1,M,r0,ier) ;find true correlations 
21 nclose=LINE(M,0,0) jE EEE EEE ESSE EEEEE ESSE EE EERE EE 

22 mseb=LINE(M,0,0) ; 

23 mseu=LINE(M,0,0) ;Reserve space and find factor 

24 fac=LINE(M,0,1) ; 

25 fac=n/{n-fac} jEREEEREEREEEEEE ESSERE EEE EERE EE 

26 ; 

27 ;startloop 

2805 

29 LIST(smp#) jtell user which sample we’re doing 

30 x=ARDT(alpha,p,1,0,n,ier,r0) ;generate sample 

31 x=x+10. ;add 10 

32 rhob=CORR(x,n,M,0,1,r0,per) ;find biased correlations 

33 rhou=rhob*fac ;multiply to get unbiased 

34 db=rhob-rhot ;difference for biased 

35 du=rhou-rhot ;difference for unbiased 


36 adb=ABS(db,M) 
37 adu=ABS (du,M) 


38 aa=adb-adu ;negative element means biased is closer 
39 aa=REPLACE(aa,M,5,0,1,0) ;this replaces negative by 1, positive by 0 
40 nclose=nclose+aa ;update number closer 

41 mseb=msebtadb~2 ;update sum of squared error 

42 mseu=mseutadu~2 

43 IF (smp#.eq.nsamps ,endloop) ;finished? 

44 smp#=smp#+1 ;no 

45 GOTO(startloop) 

46 ; 

47 ;endloop ;yes 

48 ; 

49 mseb=mseb/nsamps ;divide and display results 


50 mseu=mseu/nsamps 

51 nclose=nclose/nsamps 

S2e List 

53 LABEL(alpha)=’AR Coefficients’ 

54 LIST(alpha) 

55 LABEL(rhot)=’True Correlations’ 

56 LIST(rhot) 

57 LABEL(nclose)=’Proportion of Times Biased Closer’ 
58 LIST(nclose) 

59 LABEL(mseb)=’MSE for Biased Correlations’ 
60 LABEL(mseu)=’MSE for Unbiased Correlations’ 
61 LIST(mseb) 

62 LIST(mseu) 

63 PROMPTON 


We used this macro for the AR(2) process having coefficients 0.3 and 0.9 and 
for the AR(1) process having coefficient —0.5 and obtained the results given in 
Table 3.10. Note that the biased estimate is closer to the true correlation and 
has a smaller average squared error whenever the true autocorrelation is large, 
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Table 3.10. Example Output from the MSERHO Macro 


AR Coefficients 


ne | . 300000 .- 900000 
True Correlations 

1 | -.157895 - .852632 . 397895 -648000 -.552505 

6 | -.417448 -622489 .188957 - .616927 .015017 
Proportion of Times Biased Closer 

1 | .570000 . 790000 580000 . 710000 -650000 

6 | .680000 . 760000 .480000 . 790000 .060000 
MSE for Biased Correlations 

1 | .000526 002977 004097 .009134 011293 

6 | .013806 .021352 .016190 032362 .019104 
MSE for Unbiased Correlations 

1 | .000523 004156 004374 .011518 .013104 

6 | .015141 -026104 .014926 .039445 -015455 
AR Coefficients 

1h - .500000 
True Correlations 

1 | .500000 . 250000 .125000 062500 .031250 

6 | .015625 .007813 . 003906 -001953 .000977 
Proportion of Times Biased Closer 

1 | .600000 . 560000 . 390000 . 190000 . 100000 

6 | .070000 .030000 -010000 . 000000 . 000000 
MSE for Biased Correlations 

1 | .009776 -015815 .013685 .017076 -016424 

6 | .018038 .013897 -011512 .009919 .013561 
MSE for Unbiased Correlations 

1 | .009832 -015528 -013081 -015893 .014936 

6 | .015994 -012045 .009758 008220 .010989 


even if they are for a large lag. This was somewhat unexpected since the bias 
in the biased estimator should get larger with the lag. 


Example 3.3. | STANDARD ERROR OF AUTOCORRELATIONS 


Suppose that there is a lag g for which the true autocorrelation function 
p(v) of a time series X is essentially zero for |v| > q. Then Bartlett’s formula 


for the variance of f(v) (see part (c) of Theorem 3.1.3) can be approximated 
by 


q 
Var(a(v)) = = 1429 > 97(R)], 
Rai 


which is independent of v. In practice, one replaces the true autocorrelations 
in this expression by sample autocorrelations and then superimposes on the 
correlogram lines at plus and minus the square root of the resulting estimated 
variance. The macro given below will carry out this procedure for a user- 
specified value of g. To illustrate its use, consider Figure 3.17 which is the 
correlogram with the superimposed standard error lines for the rainfall data 
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Correlogran and Standard Error Lines 
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Figure 3.17 Correlogram and Standard Error Lines for the Rainfall Data. 


set (Series IV in Chapter 1). The STDRHO macro is appropriate for this data 


set as its correlogram is small for all lags. 


ds 

2 a3i5 STDRHO.MAC: macro to plot the correlogram of a series 
Bas of length M and superimpose the standard 

as error lines using ‘truncation point’ q (le M). 
Ss 3 

Gh Re INPUT: x, n (data and sample size) 

foe M,q 

85 

9 PAUSE 

10 ;start 


i 
i 


rho=CORR(x,n,M,0,1,r0,per) ;find correlations 


12 ss=DOT(rho,rho,q) 

13 std2=2«{{1+2ess}/n}~.5 ;find two times standard error 
14 mstd2=-std2 

15 xx=LIWE(M,0,1) ;construct input to PLOTK 

16 xx=<xx,0,M,0,M> 


i 
_ 


yy=<rho ,std2,std2,mstd2 ,mstd2> 

nn=<M ,2,2> 

type=<2,2,2> 

LABEL (yy)=’Correlogram and Standard Error Lines’ 
LABEL (xx)=’n=#n#, q=#q#? 
PLOTK(xx,yy,nn,3,type,0,M,-1,1) 


Example 3.4 INCONSISTENCY OF THE PERIODOGRAM 


NNN 
NF oO © 


In this simple macro, we generate realizations of length 200 through 1000 
in steps of 200 from a Gaussian white noise process having variance 1. For 
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each sample we graph the log of the periodogram. In each case we use 1 as the 
value to standardize the periodogram so that they are all on the same scale. 


a INCONS.MAC: macro to illustrate the inconsistency of the 


ene periodogram. 


INPUT: seed (seed for random number generator) 
+3 

PAUSE 

;start 

PLOTON 

n=200 


omar nawnwrk WYN KF 


Hee 
Ce a) 


x=WH (seed, 10) ;warm up generator 


= 
w 


J 
Ps 


;Startloop 


e 
a 


x=WH(O,n) 

rho=CORR(x,n,0,n,1,r0,per) 

LABEL(per)=’White Hoise Periodogram, n=#n#’ 

CLS 

PLOTSP(per,n,1) ;note that we’re standardizing by 1 each time 
IF (n.eq.1000, endloop) 

n=n+200 

GOTO(startloop) 


BINS SSID Re fee a eat 
mwWwNnNerF OC 06 ON DH 


nN 
an 


;endloop 


No 
for) 


’ 


PLOTOFF 


The macro FEJER.MAC calculates the Féjer kernel 


N 
a 


F,(w) = (enna 


n \ sin 7w 


for the Q equally spaced frequencies between f1 and f2 inclusive. To construct 
Figure 3.2 we used the macro twice for each n: once to get the graph for [—.1, .1] 
and once for [.1, .5]. 
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aS 

2.88 FEJER.MAC: macro to calculate the Fejer kernel for 
Siwy user-specified n for Q frequencies between 
4 se frequencies f1 and f2 (inclusive). The 
Si kernel is then plotted with the vertical 
6 scale being between 0 and ymax. 

ip isis 

9°89 INPUT: n, Q (number of frequencies between f1 and f2) 
9 ss £1, £2, ymax 

OT rss 

55 

12 PAUSE 

13 ;start 

14 freq=LIWE(Q,f1,f2,a) ;now we have frequencies 

15 ff=pi+*freq 

16 ff£=REPLACE(ff,Q,4,0,0,.0001) ;make sure frequency O not included 
17 den=SIW (ff ,Q) ; denominator 

18 ff=n*ff 

19 num=SIN (ff ,Q) ;numerator 

20 fej={num/den}~2/n 

21 LABEL(fej)=’Fejer Kernel For n=#ni#? 

22 LABEL(freq)=’ ° 

23 PLOT(freq,fej,Q,f1,£2,0,ymax) 


Example 3.6 | CDF oF THE BARTLETT STATISTIC 


The macro BARTCDF.MAC macro given below was used to find the values 
used in Figure 3.4 and Table 3.1. 


aie BARTCDF.MAC: Macro to find the cdf Pr(B.le.b) of the 
OH Bartlett statistic for b=.40,.41,...,2.00 


INPUT: Hone 

PAUSE 

PROMPTOFF ;turn prompt off 
cdf=LIWE(161,0,0) ;reserve space for cdf 


b=LIWE(161, .4,2,1) ;define the 161 values of b in [.4,2] 
i=1 


O©OMNAA HPWH KH 


a 
a —) 


, 


;start 


an 
» w 


pval=BARTTEST (b[i]) 
cdf [iJ=1.-pval 
LIST(b[i] ,cdf[i]) 
if (i.eq.161,plot) 
i=i+1 

GOTO(start) 


NNNF RRP eB 
NVrPOwHoMAN AYN 


;plot 
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23: 5 

24 LABEL(cdf)=’CDF of the Bartlett Statistic’ 

25 LABEL(b)=? ? 

26 PLOT(b,cdf,161,.4,2.0,0,1) 

27 bb=<b,cdf> ;this is how we mix two arrays 

28 bb=TRANS(bb,161,2) 

29 LABEL(bb)=’CDF of Bartlett Statistic’ 

30 LIST(bb,322,8,8f6.3) ;we could use RECORD to send this to 
31 PROMPTON 3a file. 


Example 3.7 | THE CHOICE OF m IN THE Q TEST 


The two macros given below were used to find the results in Section 3.2.1 
for studying the effect of choosing m. The macro QLEV.MAC plays the role of a 
main program, while QLEVEL.MAC is called repeatedly by QLEV.MAC. 


noe QLEV.MAC: macro to repeatedly call QLEVEL.MAC 
13 to form Table 3.2. 


INPUT: seedi, nsamps 

5 mm (array of length 3 with m’s to use) 

A nn (array of length 3 with n’s to use) 

ae aa (array of length 3 with alpha’s to use) 


OmMN ON WN 


- 
Oo 


PAUSE 

;start 

mi :=mm[1] ;note that we have to convert type 
m2 :=mm[2] ;from real to integer 

m3 :=mm[3] 

seed=seed1 


Ce a ol nol a od 
Naoaourh WN 


x=WH (seed,10) ;Wwarm up random number generator 
seed=0. 


re 
© 0 


iw] 
oO 


i=1 ;now we have a double loop over alpha and n 
381 ;we keep track of results using RECORD 
j=1 

<2 

alpha=aa[il] 

n:=nn[j] 

MACRO(qlevel,start) 

RECORD(qlevel.dat) 

LIST(alpha,n,nreji,nrej2,nrej3,ndiff) 

RECORD(close) 

IF(j.eq.3,e1) 

j=j+1 

GOTO(s2) 

jel 


wWwWwwWwWwWWoNnNNnNnNN NN NN LY 
rPwWNHrP OCHO WN AA hk WN 
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3.10 EXAMPLES AND PROBLEMS 


IF (i.eq.3,e2) 


ee QLEVEL.MAC: Macro to study the effect of m on the significance 


si INPUT: nsamps,n (number of samples and sample size) 


34 mi,m2,m3 (three different values of m (in increasing order)) 


as. OUTPUT: nreji,nrej2,nrej3,ndiff (number of rejections for 
es the different m’s and the number of times the decisions 


i=iti 

GOTO(s1) 

3e2 

; , 

+3 level of the Q test. 
>> 

33 seed (seed for WH) 

Ag alpha (nominal significance level) 
mF are different) 

+> 

PAUSE 

;start 

pv1=LIWE(nsamps,0,0) 


;reserve space for p-values 


pv2=LIWE(nsamps ,0,0) 

pv3=LIWE (nsamps 0,0) 

x=WH (seed, 10) ;warm up generator 

ns=1 ;ms counts samples 
PROMPTOFF 

;startloop 

x=WH(0,n) ;generate sample 
rho=CORR(x,n,m3,0,1,r0,per) ;find correlations 
pvi[ns]=QTEST(rho,m1,0,0,n,Q) ;find p-values 


pv2[ns]=QTEST (rho ,m2,0,0,n,Q) 
pv3[ns]=QTEST(rho ,m3,0,0,n,Q) 


LIST(ns) 

if (ns .eq.nsamps,endloop) 
ns=nsti 

GOTO(startloop) 

;endloop 

PROMPTON 

pvi=pvi-alpha 
pv2=pv2-alpha 
pv3=pv3-alpha 
x1=EXTRACT(pvi ,nsamps,0. 
x2=EXTRACT (pv2 ,nsamps,0. 
x3=EXTRACT (pv3 ,nsamps,0O. 
y1=LIWE(nsamps ,0,0) 
y2=LINE (nsamps ,0,0) 
y3=LINE(nsamps ,0,0) 
ABORTOFF 


;display sample number so that user 
;will know something is happening 


;turn prompt back on 
;subtract alpha from each vector 


0,1t,nreji) ;this pulls off indices of 
0,1t,nrej2) ;significant p-values and 
0,1t,nrej3) ;counts how many there are 
iy1,y2,y3 will be arrays of 0’s except 
;for places where there are significant 
;p-values 
jmake sure macro doesn’t abort if no rejects 
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y1=REPLACE(y1,x1,1,nreji) ;these put 1’s in the right places 
y2=REPLACE(y2,x2,1,nrej2) 


y3=REPLACE (y3 ,x3,1,nrej3) 
ABORTOE ;now any error should cause macro to abort 
y=yity2+y3 ;an element of y will be O or 3 if no difference 


x1=EXTRACT(y,nsamps,1.0,eq,ndi) ;ndi=# of 0’s 
x2=EXTRACT(y,nsamps,2.0,eq,nd2) ;nd2=# of 3’s 
ndiff=ndi+nd2 


Example 3.8 AVERAGING THE PERIODOGRAM 


The macro AVEPER.MAC was used to do the periodogram averaging proce- 


dure in Section 3.3.1. 


omananrrh OWN FP 


Te) SRS eb RR SBS RO Ob NM eee ee Se Boe Boe 
Se So ood ch Ov eos ae ee OO ©. 8 a en wk O Ne -O 


;;  AVEPER.MAC: macro to produce a smoothed periodogram estimate 
He of a spectral density. The ordinary average of 
ane 2m+i values of the periodogram is used. 
ae INPUT: per, m (periodogram and the number of frequencies on each 
cif side of the target frequencies to be used) 
PAUSE 
;start 
LENGTH(per,q) 
IF(m.eq.1) ;have to handle m=1 separately 
ft=<per[2] ,per,per[q]> 
GOTO(s1) 
ENDIF 
mp1=m+1 
pp=EXTRACT(per,2,mp1) ;put reversed values of 2nd thru (mti)st 
pp=REVERSE(pp ,m) ;at beginning 
ft=<pp,per> 
qmm=q-m 
qmmp1=qmmt+1 
Pp=EXTRACT(per,qmmpi,q) ;now reflect last m values of per 
pp=REVERSE (pp ,m) 
ft=<ft,pp> 
;s1 5 RRR ERE EEE EEE EE EEE EEE EEE EEE EE EE 
m2=2+m . 
qpm2=qtm2 ; We use the FILT command to do the 
m2pi=m2+1 ; averaging. 
wO=1. ‘ 
wt=LIWE(m2,1,0) FREESE EEE EEE REESE ERS EE EEA EEE EERE ES ES 
ft=FILT (ft ,wt ,wO,qpm2,m2) 


ft=ft/m2p1 
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Example 3.9 | SPECTRAL WINDOWS 


This macro will calculate and display any of the eight spectral windows 
given in Table 3.3 and available in the WINDOW command. The window is found 
by using the WINDOW command with a correlation sequence of ones as input. 


LAs 

2: 3% WINDOW.MAC: Macro to calculate and plot the spectral window 
aes: specified by the integer iopt (which corresponds 
405 to the list in Table 3.3) for parameters n and M. 
5 33 

Ciesis The window is calculated for Q frequencies between 
Canis O and 1 and is plotted for frequencies between 
Sine f1 and f2. The vertical scale is from ymin to ymax. 
9 55 

TOs TEPUT on; MN; adopt, Q> <1, £2. ymin, ymax 

Le TP 

12 PAUSE 

13 ;start 

14 q={Q/2}+1 ;Form frequencies 

15 freq=LIWE(q,0,.5,1) 


16 rho=LIWE(n,1,0) 

17 r0=1. 

18 w=WINDOW(rho,r0,M,Q,iopt,n) ;now have window at all frequencies 
19 wi=EXTRACT(freq,q,fi,ge,ny) EERE EEE EEE EERE REESE EEE EE EEE EEE 
20 wi=EXTRACT(w,wi,ny) : 

21 freqi=EXTRACT(freq,wi,ny) ; Now pull off values and frequencies 
22 wi=EXTRACT(freqi ,ny ,f2,1le,ny) ; between fi and f2 

23 wi=EXTRACT (wi ,wi,ny) : 

24 freqi=EXTRACT (freqi ,wi,ny) j EFEREEEEE ERASER ERE ER ERR ERE EERE EK EEE 
25 LABEL(wi)=’Window #iopt#, n=i#n#, M=#M#? 


iS) 
a 


LABEL(freqi)=? 
PLOT (freqi ,wi,ny,f1,£2,ymin, ymax) 


Example 3.10 | WinDow EsTIMATOoRS 


The macro WINDSP .MAC given below will find and plot the window spectral 
estimate (and confidence intervals) for a data set using user-specified scale 
parameter, window, and number of frequencies. The macro WINDSP3.MAC will 
form a graph of the Parzen window spectral estimator for a data set using three 
different truncation points. This macro was used to form Figure 3.8. 


i) 
J 
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ae WIBDSP.MAC: macro to find nonparametric spectral estimator 


1 

2 

Sos for a series x of length n. 

Ans’ 

5. 3 8 INPUT sex, n 

6 ns M (scale parameter), Q (number of frequencies) 
ee ieen ioptw (window to use) 

8 53 

9 PAUSE 

10 ;start 

11 mm=K ;input depends on whether truncated form is used 
12 IF(ioptw.le.5,s1) 

13 mm=n-1 


14 ;s1 

15 rho=CORR(x,n,mm,0,1,r0,per) 

16 spec=WINDOW(rho,r0,M,Q,ioptw,n,c) 

17 specl=spec/c 

18 specu=spec*c 

19 LABEL(spec)=’Window Estimator and Conf Intervals’ 
20 LABEL(specl)=’M = #M#, Window Number #ioptwt’ 

21 LABEL(specu)=<x> 

22 PLOTSP(specl,Q,r0,spec,Q,r0,specu,Q,r0) 


1 

Pte ae WINDSP3.MAC: macro to calculate Parzen window spectral estimate 
Bee for three truncation points. 

4 Si 

5.33 INPUT: x, n (data and sample size) 

Gite Mi, M2, M3 (truncation points, must be even and M3 biggest) 
ORS Q (number of frequencies) 

Sars 

9 PAUSE 

10 ;start 

11 rho=CORR(x,n,M3,0,1,r0,per) 

12 £1=WINDOW(rho,r0,M1i,Q,4) 


13 £2=WINDOW(rho,rO,M2,Q,4) 

14 £3=WINDOW(rho,r0,M3,Q,4) 

15 LABEL(f£2)=<x> 

16 LABEL(f1)=’Parzen Window Spectral Estimate’ 
17 LABEL(£3)=’Mi = #M1#, M2 = #M2#, M3 = #M3#? 
18 PLOTSP(£3,Q,r0,£2,Q,r0,f1,Q,r0) 


Example 3.11 | ESTIMATING THE INNOVATION VARIANCE 


Sometimes we would like to estimate the variance of the innovations pro- 
cess for a purely nondeterministic time series, that is, estimate 


5 
a2, = exp( [log f() dw) 
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(see part (d) of Theorem 2.4.1), given a realization X(1),...,X (n). A natural 
way to estimate o?, is by 


é Iw; 
62, = exp(— doe (un) 


where f is the periodogram. Davis and Jones (1968) have shown that this es- 
timator is in fact asymptotically biased, but that if one adds Euler’s constant 
= .57721 to the exponent, then the resulting estimator is asymptotically un- 
biased. The following macro will find this unbiased estimator of the innovation 
variance. For more information about this problem, see Hannan and Nichols 
(1977) and Pukkila and Nyquist (1985). 


1 33 

2 oe INWOV.MAC: macro to estimate the innovation variance rvar 
re by the Davis-Jones method. 

SBF 

5 ors INPUT: x, n 

65> 

i? Be OUTPUT: rvar, (Davis-Jones estimate) 

8 55 

9 PAUSE 

10 ;start 

11 x=SUBMNS(x,n,1,xbar) ;find periodogram via FFT 

12 FFT(x,n,n,1,zr,zi) 

13 fhat={zr°2+zi~2}/n 

14 fhat[1i]=1. ;this has the effect of leaving out frequency 0 
15 1fhat=LOGE(fhat ,n) 

16 lfhat=SUBMNS(1fhat,n,1,rvar,0) ;this approximates integral 

17 rvar=rvar+.57721 jadd bias correction 

18 rvar=EXP(rvar) 

19 LIST(rvar) 


Example 3.12 | YULE-WALKER AND BurRG ESTIMATORS 


As we pointed out in Section 3.4.4, the Yule-Walker estimation procedure 
can perform poorly for AR models that are ill-conditioned, that is, close to non- 
stationary. An extreme example of this is the AR(4) model having coefficients 
—2.7607, 3.8106, —2.6535, and .9238. In Figure 3.18 we give the log of the true 
spectral density of this process. Note that it contains a pair of sharp peaks 
that are quite close together. The fact that this model is ill-conditioned is 
emphasized by the fact that the zeros are given by: 


Real Part of Polynomial Roots 

1 | - 649963 - 649963 - 786224 - 786224 
Imaginary Part of Polynomial Roots 

1] - .785937 - 788937 -650041 -.650041 
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5.097 AR(4) Spectral Density 


.- 0.18 0.20 0.30 0.4 0.58 


Spectra for Yule-Walker and 
‘ og fule-dalker Fee te ee pea 


0.0 9.18 0.20 0.30 0.4 0.58 


Figure 3.18. True AR(4) Spectral Density and the Burg and Yule-Walker 
Estimates for One Realization of Length 100. 


Modulus of Roots 


Zi 1.019877 1.019877 1.020148 1.020148 


Thus the zeros are complex (which will make the data sinusoidal), close to 
the unit circle (and thus the process close to nonstationary), and also close 
together. 


The following macro will generate a series of realizations from any user- 
specified AR process, and then use both Yule-Walker and Burg estimates of its 
coefficients to estimate the true spectral density. 
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1 3% 

2 ;;  BURGYW.MAC: macro to superimpose the Yule-Walker and 
S ais Burg estimates of the spectral density of 
4 6h an AR(p) process. 

5 53 

6 sig nsamps realizations of length n are generated 
up oe from the process having coefficients alpha and 
8 Ess error variance 1. 

953 

10°35 INPUT: p, alpha, nsamps, n, seed 

11 3; 

12 PAUSE 

13 ;start 

14 ns=1 

15 x=WH(seed,10) ;warm up random number generator 

16 3 

17 ;s1 

18% 

19 x=ARDT(alpha,p,i,0,n,ier,r0) ;generate data 

20 x=SUBMNS(x,n,1,xbar) ;subtract mean 

21 alphayw=DTAR(x,n,p,1,2,1,p,r0,rvaryw) ;use YW 


22 alphabg=DTAR(x,n,p,1,2,2,p,r0,rvarbg) ;use Burg 
23 LABEL(alphayw)=’Yule-Walker Coefficients’ 

24 LABEL(alphabg)=’Burg Coefficients’ 

25 LIST(alphayw) 

26 LIST(alphabg) 

27 LIST(rvaryw,rvarbg) 

28 fyw=ARSP(alphayw,p,rvaryw , 256) ;YW spectra 
29 fbg=ARSP (alphabg ,p,rvarbg , 256) ;Burg spectra 
30 LABEL(fyw)=’Spectra for Yule-Walker’ 

31 LABEL(fbg)=’Spectra for Burg’ 

32 PLOTSP(fyw,256,r0,fbg,256,r0) ;plot 

33 IF(ns.eq.nsamps,end) 

34 ns=nst1 

35 GOTO(s1) 

36 ; 

37 ;end 

38 ; 


We tested the macro by generating five realizations of length 100 from the 
AR(4) process described above. The resulting estimates are given in Table 3.11. 
The Yule-Walker estimates are consistently poor, while the Burg estimates are 
quite close to the true values. The consistent behavior of the Yule-Walker 
estimates suggests that the Yule-Walker procedure is trying to estimate the 
coefficients of some other AR process rather than the AR(4). This is in fact 
what is happening as the closeness of the zeros of the characteristic polynomial 
to the unit circle is in essence “tricking” the Yule-Walker procedure into trying 
to fit an AR(2) model (see Findley (1978) and Quinn (1980) for more discussion 
of this issue). In Figure 3.18 we superimpose the spectral estimates for the 
Yule-Walker and Burg estimates for one of the realizations. Note that the 
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Table 3.11. Example Ouput from the BURGYW Macro 


Yule-Walker Coefficients 


1 | -1.339443 - 843000 -093859 - .031056 
Burg Coefficients 
1} 2.644507 3.580996 -2 .453292 . 864386 


rvaryw=15.646350 rvarbg=.889589 
Yule-Walker Coefficients 


1 | -1.735767 1.617713 -.571055 -119940 
Burg Coefficients 
a | -2.770745 3.834797 -2.690591 - 940820 


rvaryw=29.172470 rvarbg=.984955 
Yule-Walker Coefficients 


1 | -1.132773 -679221 - 184523 - .019664 
Burg Coefficients 
17} -2.727930 3.748685 -2.601738 . 907362 


rvaryw=68.174780 rvarbg=1.210733 
Yule-Walker Coefficients 


ia -1.714853 1.457379 - .392310 -064111 
Burg Coefficients 
a ~2.759392 3.812833 -2.657903 - 928765 


rvaryw=22.233480 rvarbg=.897821 
Yule-Walker Coefficients 


1 | -1.784269 1.583475 ~.514477 .115106 
Burg Coefficients 
11 -2.729429 3.654166 -2.452617 -811345 


rvaryw=16.139020 rvarbg=.945297 


Yule-Walker estimator fails to “split the peak,” while the Burg estimator seems 
quite good. 


Example 3.13 | PREDICTION VARIANCE HORIZONS 


omAnNraanh WH eH 


el a ool el onl 
re WN re OC 


The macro PVH.MAC can be used to find the values of PVH for a data set. 


ie PVH.MAC: macro to estimate the PVH function 


;; INPUT: x, n (data and sample size) 
5 p, m (AR order to use and maximum index for PVH) 


;; OUTPUT: pvh (array of length m containing PVH(1),...,PVH(m)) 
nes pvhO (real scalar containing PVH(O)) 


PAUSE 

;start 

mp1=m+1 

rho=CORR(x,n,m,0,1,r0,per) ;find correlations 
alpha=CORRAR(rho,r0,p,rvar) ; and AR parameters 


gama=INVPOLY(alpha,p,m) ;find MACinfinity) coefficients 
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15 gama=<1,gama> jnow find PVH 

16 gama=gama*gama 

17 pvh=CUM(gama,mp1 ,1) 

18 pvh=rvar*pvh 

19 pvh=pvh/r0O 

20 pvhO=pvh[1] 

21 pvh=EXTRACT(pvh, 2 ,mp1) ;finally have it 
22 LABEL(pvh)=’PVH Function PVH(0)=@pvh0oe’ 

23 LIST(pvh,m,10,10f6.3) 


Example 3.14 | CAT as A WHITE NolsE Test 


The macro CATWN.MAC will generate a series of white noise realizations of 
length n and then find the CAT criterion for orders 0 through M, keeping track 
of how many times each order is chosen. 


ile 

Pe Be CATWH.MAC: macro to investigate the level of significance 
ones of the CAT criterion as a test for white noise. 
4 is 

5 3; INPUT: seed, nsamps (number of samples to generate) 

6 3; n (sample size) M (maximum order) 

«35 

8 PAUSE 

9 ;start 

10 PROMPTOFF 

11 smp#=1 ;initialize sample counter 

12 x=WH(seed,10) ;warm up random number generator 

13 ords=LIWE(M,0,0) ;reserve space for chosen orders 

R445 

15 ;s1 

PGi 

17 LIST (smp#) ;tell user which sample we’re doing 

18 x=WN(0O,n) 


19 x=SUBMNS(x,n,1,xbar) 

20 x=DTAR(x,n,M,2,2,1,p,r0,rvar) ;find order 
21 IF(p.gt.0) 

22 ords[p]=ords[p]+1 3add 1 to order counter (unless p=0) 
23 ENDIF 

24 IF(smp#.eq.nsamps,e1) ;finished? 
25 smp#=smp#+1 ;no 

26 GOTO(s1) 

26% 

28 ;el1 

29 ; 

30 cords=CUM(ords ,M,1) 

31 nwrong:=cords[M] 

32 nright=nsamps-nwrong 

33 ;list 

34 LIST(n,M) 
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35 LABEL(ords)=’Number of Times Various Orders Chosen’ 
36 LIST(ords,M) 

37 LIST(nright) 

38 PROMPTON 


To illustrate the macro, we generated 1000 realizations of length 100 and 
found that 928 of the realizations chose white noise (order 0) when tested only 
versus order 1. When orders as high as 10 were allowed to be chosen, we found 
the following frequencies: 


Number of Times CAT Chose Order p: 


p | ft) 1°23 4S Oe te See 10 
# | 827 63) 32) 31 14" 17 oe 7 eae ee 


Example 3.15 | AIC FoR ARMA PROCESSES 


In the ARMAAIC.MAC macro given below, we use ARMASEL to find an estimate 
of the residual variance for AR orders 1 through pmax and MA orders 1 through 
qmax. These variances are used to find the corresponding AIC values. 


ars 

Oe ARMAAIC.MAC: macro to find the AIC for ARMA(p,q) 

3 455 orders p=1,...,pmax, and q=1,...,qmax. 
Ass 

[Oe INPUT: x, n (data and sample size) 

Gora pmax, qmax 

as 

8 ;; OUTPUT: aicpq (matrix whose (p,q) element is AIC(p,q)) 
9 33 aicOO ( AIC(O,0) ) 

10 3; 

11 PAUSE 

12 ;start 

13 pq=pmax*qmax 

14 aicpq=LIWE(pq,0,0) 

15 rho=CORR(x,n,0,0,1,r0,per) 

16 aicOO=LOGE(r0) 

17 aic00O=n*aic00 

18 p=1 

19 q=1 

20 PROMPTOFF 

21 ii=0 

22. 5 

23 ;s4 

245 

25 ii=iitl 

26 alpha=LIWE(p,0,1)  ;use ARMASEL to estimate residual variance 


N 
~ 


beta=LIWE(q,0,1) 
kopt=ptq 


iw] 
Qo 
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29 npar2=2*kopt 

30 kopt=-kopt 

31 mn=-n 

32 ARMASEL(x,mn,30,10,p,q,kopt, .95,p,q,alpha, beta, rvar, ier) 
33 LIST(alpha,p) 

34 LIST(beta,q) 

35 aic=LOGE(rvar) 

36 aic=n*aictnpar2 

37 aicpq[ii]=aic 

38 LIST(aic) 

39 IF(q.eq.qmax,s5) 

40 q=qti 

41 GOTO(s4) 

42) 3 

43 ;s5 

44 ; 

45 IF(p.eq.pmax,s6) 

46 q=1 

47 p=pti 

48 GOTO(s4) 

49 ; 

50 ;s6 

SL; 

52 LABEL(aicpq)=’AIC for ARMA(p,q)’ 
53 LISTM(aicpq, pmax ,qmax) 
54 LIST(aic00) 

55 PROMPTON 


Example 3.16 | MODEL IDENTIFICATION 


The basic aim of TIMESLAB is to provide users with enough building 
blocks so that they can produce very general, easy to use macros for analyzing 
data. The macro ID.MAC is an example of such a macro. It guides the user 
through a series of steps that should be helpful in finding transformations 
and models that can be used with other macros for spectral estimation and 
prediction. 


1 CLS ;clear the screen 

2 33 

3 33 ID.MAC: This macro will guide you through the ARMA model selection 
4°36 process. To use this macro you need to have defined 

3s Ss an array x containing your data and the integer n 

6 3y which is the sample size. 

YOM 

8 33 

9 PAUSE 

10 CLS 

11 ;; We first try to get a feel for your data. I will plot them as well 
12 ;; as their histogram and quantile plot. If either of these appear 

13 ;; to be long tailed (i.e. skewed), then a transformation is probably 
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;; called for and you should break the macro and consider some power 
33 transform. 


3; First the plot of the data. It will not have ‘nice’ tic mark 
33 labels. 


>? 

PAUSE 

PLOT(x,n,0,n,0,0) 

3; Mow the histogram: 

PAUSE 

HIST(x,n) 

+5 

si Now the quantile plot. The plot should look linear in the middle 
soy and hit the vertical axes near where they intersect the horizontal 
33 axes. 

PAUSE 


INFQHT(x,n) 


Hin If those plots looked ‘strange,’ you should break the macro now 


rie and consider transforming the data. 

PAUSE 

m=0 

CLS 

> 

;; Wow we see if the data could be white noise. We’ll plot 

he the sample correlogram with 95% simultaneous confidence 

;; bands as well as the cumulative periodogram with its 

i confidence bands. 

3; I will use m=min(n/4,30) as the number of correlations 

aie unless you break the macro, define m, and then resume execution 
3; of the macro by entering the command MACRO. I will also use 
HE n frequencies for the periodogram. 

>, 

PAUSE 

CLS 


A If the correlogram falls within the bands, your data may be 


35 white noise. 

He If the cumulative periodogram falls completely within 

;; the bands, then x might be white noise. 

a If there is an apparent discontinuity in the cumulative 

33 periodogram at a certain frequency, then either x might have 
fz a deterministic sinusoid of that frequency or x might 

are be an AR process with some roots near the unit circle. 

PAUSE 

CLS 
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68 Gs This takes a few seconds ....... 
69 35; 

70 PROMPTOFF 

71 IF(m.eq.0) ;determine m 

72 m=n/4 

73 IF(m. gt .30) 

74 m=30 

75 ENDIF 

76 ENDIF 


77 MACRO(wntest) 
78 PROMPTON 


79 CLS 

80 5; 

81 ;; Wow we’ll see if the semi-automatic methods of ARMA model 
82 ;; determination are adequate for your data or if you need to 
83 ;; identify a suitable transformation to “stationarity.” 

84 55 

85 ;; We will do this by automatically fitting an autoregressive 
86 ;; model to the data, plotting its spectral density, and 

87 ;; determining the range (max-min) of the logarithm of the 

88 ;; spectral density. This range can be used to classify the 
89 5; "memory type" of the series as short (low range) or long 
90 ;; (high range). One rule of thumb is to consider transforming 
91 ;; the data if the range is greater than 6. 

92 33 

93 ;; Henceforth I will use the next power of 2 greater than or 
94 ;; equal to n as the number of frequencies. 

95 55 

96 ;; A large dynamic range can be caused by either an infinity 
97 ;; (a peak) or a zero in the spectrum. An infinity can be 

98 ;; handled by transformation, while a zero is usually the 

99 ;; result of a transformation that caused the data to become 
100 ;; a noninvertible MA or ARMA model. If this happens, you 

101 ;; might want to undo the offending transformation and perhaps try 
102 ;; another. 

103: 

104 PAUSE 

105 CLS 

106.47, 

107 ;; This takes a few seconds .... 

108 ;; 

109 PROMPTOFF 

110 1n=LOGE(n) ;find next power of 2 greater than or equal to n 


111 le2=LOGE(2) 

112 1n={1n/le2}+.99 

113 Q:=ln 

114 Q=27Q 

115 q={Q/2}+1 

116 rho=CORR(x,n,m,0,1,r0,per) 
117 alpha=CORRAR(rho,r0,m,n,p,rvar,cat) 
118 f=LINE(q,1,0) 

119 IF(p.gt.0) 

120 f£=ARSP(alpha,p,rvar ,Q) 
121 f=f/ro 
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ENDIF 

££=LOGE(f ,q) 

MAXMIN (ff ,q,fmax, imax ,fmin,imin) 
rge=fmax-fmin 

LABEL(f)=’AR Spectra, p=#p#, range=@rge@’ 
PROMPTON 

PLOTSP(f ,Q,1) 


3; Wow we will assume that your data either are short memory or 
ie: have been transformed so that they are short memory. 


$ We now look at a variety of diagnostics that will suggest 
; one or more ARMA type models for your data. 


;; We begin with the sample correlogram. For an MA model this 
33 will be zero past some lag. For an AR or ARMA model it 

ats decays to zero and stays there, or else looks like a damped 
ee sinusoid. 

PAUSE 

CLS 

rho=CORR(x ,n,m,0,1,r0,per) 

zero=LINE(m,0,0) 

LABEL(zero)=’ ” 

PLOT2(rho,zero,m,m,3,3,0,m,-1,1) 

35 Next is the partial correlogram. For an AR process this 

uk will be essentially zero past some lag. 

PAUSE 

CLS 

y=SUBMS (x ,n,1,xbar) 

33 This takes a few seconds ...... 

part=PARCORR(y ,n,m,brv) 

PLOT2(part ,zero,m,m,3,3,0,m,-1,1) 


an Next are the biased residual variances. For an AR process, 
one these will decrease until the correct order at which point 
3; they remain roughly constant. We will plot the standardized 
i version of the biased residual variances. 

PAUSE 

brv=brv/r0 

LABEL(brv)=’Standardized Biased Residual Variances’ 
PLOT2(brv,zero,m,m,3,3,0,m,0,1) 

He Wext are the standardized unbiased residual variances. 

| For an AR process, the minimum of this function could 

on be the AR order. 

PAUSE 

nmj=LIWE(m,0,1) 

urv=n*brv/nmj 
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LABEL (urv)=’Standardized Unbiased Residual Variances’ 
PLOT2(urv,zero,m,m,3,3,0,m,0,1) 
;inverse 


yc Wow we will consider the above quantities for the 
33 process corresponding to the inverse spectral density. 


; The interpretation of these quantities is the reverse of 
He those for f. 


3; First the inverse correlogram. Wow a truncated correlogram 
as indicates that x may be an AR process. 

+5 

PAUSE 

rho=CORR(x,n,m,0,1,r0,per) 
alpha=CORRAR(rho,r0O,m,rvar) 

rvari=1/rvar 

rhoi=MACORR(alpha,m,rvari,m,riO) 
LABEL(rhoi)=?’Inverse Autocorrelations’ 
PLOT2(rhoi,zero,m,m,3,3,0,m,-1,1) 

>” 

:3 Next are the inverse partials, biased residual variances, 
a and unbiased residual variances: 

PAUSE 

alpha=CORRAR(rhoi,riO,m,rvar) 

part=ARPART (alpha,m, ier) 

LABEL (part)=’Inverse Partials’ 

PLOT2(part ,zero,m,m,3,3,0,m,-1,1) 

al=<-1> 

lpart=1-part~2 

lpart=loge(lpart ,m) 

lpart=<0,lpart> 

mp1i=m+1 

brv=ARDT (al ,1,mpi,1part) 

brv=EXTRACT (brv, 2,mp1) 

brv=EXP (brv ,m) 

LABEL (brv)=’Inverse Biased Residual Variances’ 
PLOT2(brv,zero,m,m,3,3,0,m,0,1) 

urv=n*brv/nmj 

LABEL(urv)=’Inverse Unbiased Residual Variances’ 
PLOT2(urv,zero,m,m,3,3,0,m,0,1) 

¢ Now we use the AIC to look for the best full 
3 AR model: 


jaic 

PAUSE 

y=SUBMBS (x ,n,1,xbar) 

+5 

“a4 This takes a few seconds ..... 
>” 


part=PARCORR(y ,n,m,brv) 
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aicar=loge(brv,m) 
aicar=aicar*n 
ajj=Line(m,0,2) 
aicar=aicartajj 
LABEL (aicar)=’AIC’ 
aicO=LOGE(r0) 
aicO=n*aicO 

CLS 

LIST(aicar) 
LIST(aicO) 

PAUSE 

j;armasel 

CLS 


a Wow ARMASEL to look for Subset AR. We’1l do it twice, 
3; the first time allowing the possibility of more lags being 
ae included than the second time. 


rie pval is .5 for the first one 
PAUSE 
PROMPTOFF 
s=10 
k1i=m-s 
k2=0 
rvar=ro 
ARMASEL(y,n,m,s,ki,k2,0,.5,p,q,alpha, beta,rvar,ier) 
npar2=0 
IF (p.gt.0) 
LIST (alpha, p) 
aa=EXTRACT(alpha,p,0,eq,npar2) 
npar2=p-npar2 
npar2=2*npar2 
ENDIF 
aici=loge(rvar) 
aici=n*aici+npar2 
LIST(aici) 
PROMPTON 
> 
nS, Wow the second time (pval=.95): 
PAUSE 
PROMPTOFF 
rvar=roO 
ARMASEL(y,n,m,s,k1,k2,0,.95,p,q,alpha,beta,rvar,ier) 
npar2=0 
IF(p.gt.0) 
LIST(alpha,p) 
aa=EXTRACT (alpha,p,0,eq,npar2) 
npar2=p-npar2 
npar2=2*npar2 
ENDIF 
aici=loge(rvar) 
aici=n*aici+npar2 


CH. 
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284 LIST(aici) 

285 PROMPTON 

286 PAUSE 

287 ;armaaic 

288 CLS 

289 ;; 

290°; Now we will look for ARMA models: 
298 25 

292 ;; First we use ARMASEL to get AIC for each combination of p and q 
293 ;; between 1 and 5 

294 «5 

295 3; This takes a while and you will see each model. 
296 33 

297 PAUSE 

298 PROMPTOFF 

299 pmax=5 

300 qmax=5 

301 MACRO(armaaic,start) 

302 PROMPTON 

303 PAUSE 

304 ;gpac 

305 CLS 

306 ;; 

307 6 Now we find the GPAC array for maximum AR and MA orders 
308: 3; being 7. This also takes a while. 
309 3% 

310 PAUSE 

311 PROMPTOFF 

312 pmax=7 

313 qmax=7 

314 MACRO(GPAC) 

315 PROMPTON 

316 PAUSE 

317 ;armasub 

318 CLS 

319 ;; 

320 ;; Finally, we will let ARMASEL suggest a subset ARMA model 
321 ;; using pval=.5 

322 3% 

323 PAUSE 

324 PROMPTOFF 

325 rvar=r0 

326 mmi0=m-10 

327 ARMASEL(y ,n,m,10,mmi0,mmi0,0,.5,p,q,alpha, beta, rvar, ier) 
328 LIST(p,q) 

329 npar2=0 

330 IF(p,6,6,1) 

331 aa=EXTRACT(alpha,p,0,eq,np2) 

332 np2=p-np2 

333 np2=2*np2 

334 npar2=npar2+np2 

335 LIST(alpha,p) 

336 IF(q,6,6,1) 

337 aa=EXTRACT(beta,q,0,eq,np2) 
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338 np2=q-np2 

339 np2=2*np2 

340 npar2=npar2+np2 
341 LIST(beta,q) 
342 aici=loge(rvar) 
343 aici=n*aic1 

344 aicl=aici+npar2 
345 LIST(aici) 

346 PROMPTON 

347 PAUSE 

348 CLS 

349 5; This is the end of the model identification macro. 


Example 3.17 | FISHER’s EXACT TEST 


Given a data set, the macro FISHER.MAC given below will return the value 
of the test statistic, the frequency for which it occurs, and the p-value for the 
test statistic. It calls the macro FISHPV.MAC to find the p-value. 


aris 

2 sig FISHER.MAC: Macro to perform Fisher’s exact test for 
Eipete the presence of a deterministic sinusoid. 
Agi 

oa INPUT: x,n 

6 +> 

7 ;;  OUPUT: g (value of statistic), freq (peak frequency), 
Sins pval (pvalue of g). 

933 

10 PAUSE 

11 ;start 

12 y=SUBMNS(x,n,1,xbar) 

13 q={n/2}+1 

14 FFT(y,n,q,1,zr,zi) 


uo 


per={zr~2+zi~2}/n 


16 MAXMIN(per,q,g,imax,fmin,imin) j;g is now max(per) 
17 freq={imax-1.}/n ;now we’ve got freq 

18 per=CUM(per,q,1) ;crude way to get denominator 
19 g=g/per[q] ;now we’ve got g 

20 MACRO(fishpv,start)  ;find p-value 

21 LIST(pval,freq) 


;; FISHCDF.MAC: Macro to find p-value of Fisher’s exact test for 
4 specified n and g 


7 INPUT: n,g 


Ge OUPUT: pval 


oOnNaunhk WHF 
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PAUSE 

;start 

m=n/2 

k:=1/g sk is [1/g] 
LIST(g,k) 


: Find p-value: 


3; Calculating p-value ... 

a) 

PROMPTOFF 

j=1 

pval=0.0 ;initialize pval 


;startloop 


LIST(j) 

pp=BIHOM(m, j) 
pval=pval+{{-1.}~{j-1}}+*pp*{1-j+*g}~{m-1} 
IF(j.eq.m,endloop) 

IF(j.eq.k,endloop) 

j=j+1 

GOTO(startloop) 

;endloop 


PROMPTON 


PROBLEMS 


Example 3.18 | CONFIDENCE BANDs For AR SPECTRA 


The macro LYNXCB.MAC was used to form Figure 3.14. It is written specif- 


ically for the lynx data but can be easily modified for other data sets. 


CMONAN HK WN KE 


ll eal al oe ed ed 
NAA bh WHF OO 


ake LYHXCB.MAC: macro to find confidence bands for the spectral 
+3 density for the lynx data 


33 INPUT: conf (level of confidence) 
PAUSE 

;start 

READ (lynx,x,n) 


;starti ;entry point if we already read data 


y=LOGE(x ,n) 
ee=LOGE(10) 


y=y/ee ;now y has logiO of lynx data 


y=SUBMNS (y ,n,1,ybar) 
alpha=DTAR(y,n,20,2,2,1,p,r0,rvar) 
f=ARSP(alpha,p,rvar ,128) 
ARSPCB(alpha,p,rvar ,n,128,conf,f1l,fu) 


;use CAT to choose order 
;point estimate 
;find bands 
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: Note how we use LABEL command. In PLOTSP, the label 
; for the last array goes on top. 

nc :=100*conf 

LABEL(£1)=’AR spectral estimator and #nc#% bands’ 
LABEL(f)=’for lynx data’ 

LABEL (fu)=’ 2 

PLOTSP(f ,128,r0,f1,128,r0,fu,128,r0) 


Example 3.19 | ESTIMATING PEAK FREQUENCIES 


CANAAN PWN 
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The macro LYNXPEAK.MAC was used to form Figure 3.16. 


ie LYNXPEAK: macro to find the confidence interval for the 
Hn period for the lynx data and produce plots of 
site the logiO of the data and the AR(11) spectra 
as with the values for the confidence interval 
33 placed on the plot. 


a INPUT: none 

PAUSE 

READ (lynx,x,n) 

y=LOGE(x ,n) 

ee=L0GE(10) 

y=y/ee 

year=LIWE(n,1820,1) ;this gets array 1821,...,1934 
LABEL (y)=’1logi0O of lynx data’ 

LABEL(year)=’ ” 

PLOT(year,y,n,1820,1940,1,4) 

y=SUBMES (y ,n,1,ybar) ;subtract mean 
alpha=DTAR(y,n,11,1,2,1,p,r0,rvar) ;fit AR(11) 
ARSPPEAK(alpha,p,rvar,n,ier,freq,se) ;find peak frequency and std error 
se2=2.*se 


fup=freqtse2 

flow=freq-se2 ;flow and fup are limits on frequency 
plow=1./fup 

pup=1./flow ;plow and pup are limits on period 


f=ARSP(alpha,p,rvar,256) ;AR(11) spectral density 
LABEL(f)=’Lynx data, Period CI:(@plow@,@pup@)’ ;form plotting label 
PLOTSP(f ,256,r0) 


Example 3.20 | AUTOREGRESSIVE SPECTRAL ESTIMATION 


The macro ARSP.MAC will find the autoregessive spectral estimator for a 


data set using either a specified order or the order that the CAT criterion 
chooses. This macro was used to form Figure 3.13. 
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sia ARSP.MAC: macro to perform autoregressive spectral estimation. 


oH INPUT: x, n (data and sample size) 
A iopt ( 1 means fit order m, 2 means fit CAT order .le. m) 
33 m (order or maximum order to use) 


oe Q (number of frequencies) 
PAUSE 

;start 

y=SUBMBS (x ,n,1,xbar) 


alpha=DTAR(y,n,m,iopt,1,1,p,r0,rvar,cat) 
IF (p.eq.0, end) 

f=ARSP(alpha, p,rvar ,Q) 

LABEL (f) =<x> 

PLOTSP(f ,Q,r0) 

pend 


Example 3.21 | PROPERTIES oF AR SPECTRAL ESTIMATION 


The macro ARSPSM.MAC generates a series of realizations from a user- 


specified autoregressive process and superimposes on one set of axes the au- 
toregressive spectral estimates from each realization. Using this macro for a 
variety of types of processes can give insight into the behavior of the estimator. 


o©mMON AA PWN 
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>? 

>;  ARSPSM.MAC: macro to illustrate sampling properties of 
ae, AR spectral estimation. 

ae The AR spectral estimates for nsamps samples 
33 of size n are superimposed. 

a5 

Ae INPUT: p, alpha (true order and coefficients) 

ie nsamps, n, maxp (number of samples, seed, maximum order) 
is seed (randon number generator seed) 

PAUSE 

;start 

PLOTON ;switch to graphics mode 

PLOTSIZE(0) ;use whole screen for graphs 

BATCHON ;don’t pause between graphs 

x=WH(seed,10) warm up generator 

ns=1 ;initialize counter 

;startloop 

x=ARDT(alpha,p,1,0,n,ier,r0) ;generate data 
x=SUBMNS(x,n,1,xbar) ;subtract mean 
alphai=DTAR(x,n,maxp,2,2,1,pi,r0,rvar) ;fit AR 


f=ARSP(alphai , pi ,rvar,256) ;find spectra 
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Figure 3.19. The Results of the Autoregressive Filtering Example. 


26 LABEL(f)=’Sample #ns# of #nsamps#’ 
27 PLOTSP(f ,256,r0) 
28 IF(ns.eq.nsamps,endloop) 


;tell user which sample 


;done all samples? 


29 ns=ns+1 ;no 
30 GOTO(startloop) 

Spy 

32 ;endloop ;yes 
Boiss 


34 GRMENU(O,.5,-6,6) 
35 BATCHOFF 
36 PLOTOFF 


AUTOREGRESSIVE FILTERING 


We saw in the bandpass filter example in Section 2.3 that it is possible to 
remove frequency components from a time series by applying what is essentially 
a moving average smoother to the data. This smoother can be thought of as a 
nonparametric method as it uses none of the properties of the data and assumes 
no model for it. We also saw that such smoothers can lead to distortion of the 
spectral density in other frequency ranges. 

One solution to this problem is to apply an autoregressive filter to the data. 
For example, consider the wave data in Figure 3.19. This data set consists of 
the forces on a cyclinder suspended in a tank of water. The water in the tank 
has waves with period approximately 2 seconds. The sampling interval for the 
forces recorded on the cylinder is 0.15 second. Thus we would expect to see 
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a periodicity in the data of approximately 15 time periods. Unfortunately, 
the instrument used to measure the forces on the cylinder induced some high 
frequency oscillation onto the plot of the data. Thus the data appear to be 
fairly regular in their 2-second oscillation, but also have a rapid oscillation. The 
researchers analyzing the data removed this high frequency oscillation using a 
low pass filter but were dissatisfied with the amount of distortion caused by 
the filter. 


We applied the ARSP macro with iopt=2 and m=20 and found that an 
AR(10) process with the coefficients given in Table 3.12 best fit the data. We 
also give the roots of the AR(10) polynomial (as found by the POLYROOTS 
command) and their moduli in Table 3.12. 


Table 3.12. AR Coefficients and Roots for the Wave Data 


Autoregressive Coefficients 


-1.24154 - 429067 ~.078211 - 455949 — .463727 
- 337943 - 099405 - .397343 -445400 -.159155 
Real Part of Polynomial Roots 
-1.23660 - .723344 ~ .723344 .04786 .04786 
-82552 -825529 - 947287 - 94728 1.84047 
Imaginary Part of Polynomial Roots 
- 000000 - 734836 ~ . 734836 -1.23346 1.23348 
- 951482 - .951482 - .420146 -420146 -00000 
Moduli of Roots 
1.23660 1.03112 1.03112 1.23441 1.23441 
1.25969 1.25969 1.03628 1.03628 1.84047 


In Figure 3.19 we give the resulting autoregressive spectral estimator. Notice 
that it clearly demonstrates the wave oscillation and high frequency noise con- 
tained in the data. Inspection of the roots shows that two of the complex pairs 
are close to the unit circle, the first being roots 2 and 3, and the second being 
roots 8 and 9. Thus we formed two sets of AR(2) coefficients via the ROOT- 
SPOLY command and formed a time series Y, by applying the first AR(2) filter 
to the original data, and a series Y2 by applying the second AR(2) filter to the 
original data (the ARFILT command was used with rvar=1). These two series 
were then each analyzed by the ARSP. Order eight was chosen in each case. 
The resulting autoregressive spectral estimates are given in Figure 3.19 which 
shows that indeed we have identified the parts of the model corresponding to 
the two peaks in the original spectral density. Notice how little distortion is 
evident in the spectral densities of the filtered data sets. 
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Computational Problems 


C3.1. For a realization of length 100 from the AR(2) process with coefficients 
0.3 and 0.9 and error variance o? = 1, what is the equivalent number of inde- 
pendent observations for estimating 4? (Hint: Use ARCORR and ARSP to find 
R(0) and f(0), respectively.) 


C3.2. Theorem 3.1.1 shows that if X is a covariance stationary time series with 
summable autocovariance function, then the variance of X, goes to zero at the 
rate of 1/n; that is, for large n, nVar(X,) is roughly constant, with limiting 
value equal to f(0) if the spectral density f of X exists and is continuous 
at frequency 0. Tsay (1986) shows that if X is a random walk process with 
independent, identically distributed errors ¢, then the variance of X, in fact 
goes to infinity at the rate of n; that is, n~!Var(X,,) is roughly constant, with 
limiting value o?/3, where o? is the variance of the error series €. Verify this 
fact by generating 500 realizations from a Gaussian random walk having o? = 1 
for each of sample sizes n = 100, 200, and 400, calculating the sample variance 
for each of the three sets of 500 sample means, and then dividing each of these 
sample variances by n. Each of these three estimates of n~Var(X,) should 
be approximately 1/3. (Hint: See the MEDMN macro for an example of writing 
a simulation macro.) 


C3.3. The Bartlett test for white noise relies on the fact that the cumulative 
periodogram of white noise behaves like the ordered values of a sample from a 
U(0,1) population. Write a macro that will generate nsamps samples of size n 
from a U(0,1) population and superimpose the plot of the ordered values of the 
samples on the same axes. Use the SORT command to find the ordered values. 
Have the values of nsamps and n be input to the macro and make sure that 
the vertical axis ranges from zero to one. Try the macro for nsamps=100 and 
n=100. Roughly speaking the resulting graph consists of a set of bands within 
which a uniform white noise series should fall. Is the band of roughly constant 
width on the graph? 


C3.4. What are the p-values for the test of white noise for the rainfall data 
(Series IV in Section 1.1) by the BARTTEST and QTEST commands? Do they 
reach the same conclusion about whether the rainfall data come from a white 
noise process? 


C3.5. Smooth the periodogram of the artificial data set described in Section 
1.1 using the AVEPER macro with 1, 2, and 3 for m. What happens to the peak 
in the periodogram at frequency 0.10 as m increases? 
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C3.6. Evaluate the lag window generators for the Bohman and Tukey windows 
at the 101 equally spaced values between 0 and 1 inclusive, and use the PLOTK 
command to superimpose the plots of the two generators versus the values at 
which they were evaluated. Use a line plot for the Tukey window generator 
and a point plot (with lower triangles as the plotting symbol) for the Bohman 
window generator. Which generator seems to decay faster in the neighborhood 
of u=0? 


C3.7. Use the ARMASEL command to choose a subset AR model for the Wolfer 
sunspot data and then use the output of ARMASEL as the input to the SEASEST 
command to obtain maximum likelihood estimates of the coefficients of the 
model. Finally, use the SEASPRED command to find predictors and prediction 
limits for the next 24 values of the series. 


C3.8. Generate five realizations of length 100 from the AR(A4) process discussed 
in Example 3.12 and use the LSAR macro to find parameter estimates for each 
realization. Are the results more similar to the Burg estimates or to the Yule- 
Walker estimates found in Example 3.12? 


C3.9. Apply the ARSPPEAK command to the Wolfer sunspot data in an attempt 
to find a confidence interval for the so-called sunspot cycle. Use the subset AR 
model chosen in Problem C3.7. 


C3.10. Use the MSERHO macro with any AR(3) model of your choice. Do the 
results behave similarly to those in Example 3.2? 


C3.11. Perform the experiment described in Section 3.2.1 and see if you get 
similar results. 


C3.12. Generate a sample of size 200 from an MA(1) process having 3 = 2 
and o? = 1 and then use the INNOV macro to estimate ¢2. Explain why this 
estimate is close to 4 instead of 1. (Hint: See the discussion of nonidentifiability 
of MA processes in Section 2.5.3.) 


C3.13. Find and plot f’(w) for the AR(2) process having coefficients .3 and 
-9 and error variance 7? = 1. (Hint: The second derivative of the reciprocal of 
f is easy to evaluate; see Section 3.9.2.) 


C3.14. Show that the AIC chooses order 11 for the log (base 10) of the lynx 
data (use the AIC macro), while the error variance estimate obtained from 
ARMASEL for the subset model having lags 1, 2, 4, 10, and 11 has a smaller AIC 
value than any of those of the full AR models. 
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C3.15. How large must n be before loglogn is greater than or equal to 3? 
The log here refers to the natural log. 


C3.16. Use the PLOT2 command to superimpose the graphs of logn/n and 
ni/5 /n for n = 10,...,100. Have the horizontal axis range from 10 to 100 and 
the vertical axis range from 0 to 0.25. What do these two functions look like 
fori ineeer LOG 


C3.17. Use the ARSPPEAK command to find the location of the peak frequency 
in the true AR(5) spectral density in Figure 3.5. 


C3.18. Use the PVH.MAC macro to find and graph the PVH for the first and 
12th difference of the log of the sales data. What does this tell us about whether 
an MA(11) or MA(12) lag should be used in the model for these data? 


C3.19. Write a macro that will generate a user-specified number of realiza- 
tions of a user-specified length from a Gaussian white noise process. For each 
realization, use the AIC to determine if order 0 or 1 is appropriate (be clever 
here as how long the macro takes to finish depends on how you do this part). 
Test the macro by generating 500 realizations of length 100. What proportion 
of the time is order 1 (incorrectly) chosen? How does this compare with the 
asymptotic result given in Problem T3.14? 


C3.20. What does the spectral density of the AR(12) process having aj = —.9 
and all other coefficients zero look like? 


C3.21. Find the first 20 partial autocorrelations of an MA(1) process having 
coefficient .7. Is there any pattern in the partials? 


C3.22. Use the ID macro for the data set that you are analyzing from your 
field of interest. Were any preliminary transformations necessary? What model 
did you find for the data? 


C3.23. Use the ARSPPEAK command to estimate the peak frequency in the 
spectral density of each of the two LH time series. Use the standard errors of the 
two estimators to test whether the two frequencies are significantly different. 


C3.24. Use the ARSP macro described in Example 3.20 to show that there is 


no clear peak at the sunspot cycle frequency in the critical radio frequencies 
data 
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for p= 14. 


Theoretical Problems 
Se ee 


T3.1. Let X, be the sample mean of a realization Xn = (X(1),...,X(n))? 
of length n from a covariance stationary time series X having autocovariance 
function R. 

a) Show that X, = h?X,,, where h, = (A/njom pin) 

b) Use Theorem A.4.1 to show that 

" 1 n n 
Var(Xn) = > 7D) Rls — Al) 
j=lk=1 


and thus show that 


where 


c) For n > 0, the nth partial sum of the sequence R(0), R(1),... is given 
by 


while for n > 1, the nth Cesaro sum is given by the average of the first n partial 
sums, that is, by 


Show that C, = T,. 


d) It can be shown (see Anderson (1971), p. 460 for example) that a 
summable sequence is also Cesaro summable and that the two sums are the 
same; that is, if limpoo Sp < 00, then limy_4o Cy, < 00 and the two limits are 
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equal. Use this fact and parts (a), (b), and (c) above to finish the proof that 
if R is summable, then 


co 


Jim, nVar(Xp) = py R(v). 


v=-0O 


T3.2. If the coefficient of an AR(1) process having mean yp is very close to 
one, why do we need only two observations in order to have a very good idea 
of what p is? This phenomenon is an example of what are called “antithetic 
variables” (see Fishman (1973)). 


T3.3. Let {X,,n = 1,2,...} be a sequence of random variables. Use the 
continuous function theorem to show that if X, ~ AN(y, 7”), then log X, ~ 
AN(log pc), where c does not depend on p. 


: Sees 1 4 
T3.4. Show that if X has a x3 distribution, then Y = a* has the exponential 
distribution with mean one. 


T3.5. Let X(1),...,X(n) be a realization from a Gaussian WN(o”) process. 
Let w; and w, be two different natural frequencies, neither of which is 0 or 
.5. Use Theorem 1.4.1 and parts (b), (e), and (f) of Theorem A.4.2 to show 
that 2f(w;)/o? and 2f(w,)/o? are independent, identically distributed as x2 
variables. 


T3.6. Show that the unbiased estimate R is not positive definite for the real- 
ization X = (1/2,—1/2)?. (Hint: Show that the matrix TOEPL(R(0), R(1)) 
is not positive definite.) 


T3.7. Use part (c) of Theorem 3.1.3 to show that for an AR(1) process, 
: 71) 2(9)) — ite 
lim nCov(A(1), A(2)) = 2e(1 — p*), 


where p = —a. 


T3.8. Let X be an ARMA process. Show that the coefficients y of the MA(oo) 
representation of X satisfy a difference equation. Thus conclude that they are 
bounded by a geometrically decreasing sequence, and thus so do the corre- 
sponding autocovariances. Finally, use these facts to show that the spectral 
density is p-differentiable for all integers p. 
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T3.9. Show for a Gaussian AR(1) process having coefficient a that 


' 2 a? 
Var( (0) = EO) Eta 


From this, find an expression for the equivalent number of uncorrelated ob- 
servations for estimating R(0). (Hint: The asymptotic variance of the sample 
variance for a random sample of size N from a population having variance R(0) 
is 2R?(0)/N.) 


T3.10. Show that if the lag window k,(v) is an even function of v, then the 
corresponding spectral window is real and periodic of period 1 and is symmetric 
about 0. 


T3.11. Let fr (w) and frp (w) denote nonparametric spectral estimators using 
the same truncation points and the Tukey and truncated periodogram windows 
respectively. 


a) Show that 


fr(w) = 0.54 frp(w) + 0.23 frp (. _ at) + 0.23 frp (. i am) 


b) Thus show that if we calculate the two estimators at the frequencies 
w; = (j — 1)/Q, for j = 1,...,[Q/2]+ 1, and 2M divides Q, then the Tukey 
estimator at frequency w; is a weighted average of the truncated periodogram 
estimator at w; and the frequencies Q/2M before and after w;. Thus if 2M = 
Q), the Tukey estimator at a certain frequency is the weighted average of the 
truncated periodogram at that frequency and the frequencies on either side. 


T3.12. Let X(1),...,X(n) be a realization of length n from a zero mean, co- 
variance stationary time series X which has a spectral density f. Suppose that 
X = 0, and consider dividing the data into K possibly overlapping segments, 
each of length L, with the starting points of consecutive segments being D time 


units apart: 
Ast) SX) 


X(t) = X(D + t) 


Xx(t) = X((K -1)D +2), 
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fort =1,...,L and (K —1)D+L=n. Define the K series Yi,..., YK, each 
of length L, by 


Y;(t)=W(t)X;(t),  t=1,...,L, 


where W(1),..., W(L) is a weight function called a data window. Next, define 
the modified periodogram f; of Y;(1),...,Y;(L) by 


7 2 


= 1 —2ni(t—1)w 
file) = FF Lie ae 


where U = Esa W?(t), at the frequencies w, = (k—1)/L, fork =1,... pe | 


1, and form the spectral estimator f of f by the average of these modified pe- 
tiodograms of the K segments: 


mn 1 oe 
flue) = Do Siler); k=1,...,[L/2]+1. 


Note that this method of spectral estimation requires only the Fourier trans- 
form of segments of the data and not the entire data set (see Welch (1967) for 
more information about this method). 


a) If L = D and n= KL, that is, the segments are not overlapping, and 
W(t) = 1 for all t, then show that 


L 


Fwr)= Do (1- A) R(v)e~27*¥*. 


v=-L 


that is, the resulting estimator is the same as that obtained using the Bartlett 
window except that the unbiased sample autocovariances are used. Note that 
this is the original motivation that Bartlett (1950) used in proposing his estima- 
tor, that is, to estimate f by the average of periodograms over nonoverlapping 
subsets of the data. 


b) Show that if 


then f (w.) corresponds to a window estimator whose spectral window is ap- 
proximately the same as the Parzen window. 
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T3.13. Let x and y be two n-dimensional vectors. Show that the value of a 
that minimizes 


n 


S(a) = ) [2s — ays)? + So(ui — az)? 


sa 
is given by 


n 
So zy: 


t=1 


2 bs “PDS? “| 
i=l (rah 


a= 


T3.14. Let X(1),...,X(n) be a realization of length n from a Gaussian white 
noise process. 


a) Show that AIC(0) is less than AIC(1) if and only if 
np <n (1 — gat) ; 


where f” is the sample autocorrelation coefficient of lag one. 


b) Thus for large n, show that using AIC as a test for white noise versus the 
alternative of first-order serial correlation has Type I error probability 0.8427. 
(Hint: Show that nf? ~ yj and that lim,,.. n(1—e72/") = 2. You can find 
Pr(xj < 2) via the DIST command.) 


T3.15. Let X ~ AR(co) whose coefficients satisfy a; < cp* for some p € 
[—1, 1]. Let pp = logn. Show that conditions (ji), (ii), and (iii) in part (a) of 
Theorem 3.8.3 are satisfied for this choice of p. 


T3.16. If we let g(,) denote the the rth largest value of the cumulative pe- 
riodogram, then Grenander and Rosenblatt (1957) have shown under the null 
hypothesis that X is Gaussian white noise that 


 (=1)9-"(1 = ja)" 


n! 
>) Gy de Fm = 


where a = [1/z]. Show that this reduces to Fisher’s exact test when r = 1. 
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T3.17. Find the coefficients of the (nonstationary) ARMA model for X in the 
model 


g(L)G(L)W(t) = A(L)A(L)e(t), 


where W(t) = (1 — L)?X(t), g(L) = 1, G(L) = 14+ .5L?, A(L) = 1—.7L, and 
A(L)ak 


T3.18. Suppose that X is a zero mean, covariance stationary time series with 


autocovariance function Rx. Suppose that we do not observe X directly, but 
rather an “amplitude modulated” (see Parzen (1963b)) version of it: 


Y(t) =9(t)X(t), 


where g is a bounded, nonrandom function such that 
1 n-—v 
R,(v) = Jim — > g(t)g(t + v) 


exists for all v € Z. 


a) Show that Y is not covariance stationary but is asymptotically covari- 
ance stationary in the sense that if we define 


Aveie LS ¥(yvt ao 
t=1 


then limy-+oo E(Ry (v)) exists and is given by 
Ry(v) = Rg(v)Rx(v): 


Note that Parzen (1963b) has shown that for any lag v for which R,(v) 4 0, 
Ry(v)/R,(v) is a consistent estimator of Rx (v). 


b) Now suppose that we have a realization X(1),...,X(n) from X, and 
that some of the data points are missing. Define 


1, if X(t) is not missing 
g(t) = { 
0, if X(t) is missing. 


Consider the case of systematically missing values; that is, we observe m values, 
then k are missing, then we observe m more, and so on. Note that this situation 
arises in any time series that periodically cannot be observed, for example, a 
medical data set where observations are not made on a patient overnight. Show 
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that if m > k, that is, more data are observed than are missed, then Ry is 
periodic of period m + k, and 


_ max(v,m) — min(k, v) 


= = Pe, k. 
R,(v) aay : v— 0,1, ....,77 + 
c) Now suppose that we have missing data and g(1),-..,9(n) are iid binary 


random variables, that is, Pr(g(i) = 1) = p and Pr(g(#) = 0) =q =1-—>p, and 
the g’s are independent of the X’s. Show that if X is a Gaussian process, then 


is a consistent estimator of Rx(v), where 
A 1 n—v 
Flv) = 5 Lo altalt + ») v=0,1,...,n—1. 


(Hint: The weak law of large numbers can be used to show that R,(v) Rls jie 
while Isserlis’s formula (see above Theorem 3.1.3) can be used to show that 


Ry(v) ™*s p?Rx(v), which means that Ry (v) > p’Rx(v) since convergence 
in mean square implies convergence in probability.) Given consistent estimators 
of Rx, we can use them to do nonparametric spectral estimation, to fit ARMA 
models, and so on. 


CHAPTER 4 


Analyzing Bivariate Time Series 


In this chapter we consider the analysis of data from a bivariate time series 
{X(t),t € Z}. Thus at each time point, we have observations X(t) and X2(t) 
on two phenomena, and we write X(t) = (Xi(t), X2(t))7. Sometimes we will 
think of the data as a single realization from a vector time series, and at other 
times we will consider that we have a pair of univariate data sets. We will try to 
emphasize mostly the ideas and methods that are new to the bivariate situation, 
particularly in the problem of determining what information is contained in one 
of the univariate series that helps to explain the behavior of the other. 


4.1. Probability Theory for Bivariate Series 


We begin our discussion by considering what is meant by bivariate covari- 
ance stationarity. 


4.1.1. Covariance Stationarity 


Definition. The bivariate time series {X(t),t € Z} is said to be covari- 
ance stationary if the univariate series {X,(t),t € Z} and {X2(t),t € Z} 
are covariance stationary in the univariate sense, and there exists a sequence 
{Ri2(v),v € Z} such that 


Cov (X1(t), X2(t + v)) = Rio(v), for all t. 


The sequence Rj» is called the cross-covariance sequence of X. We denote the 
autocovariances of X, and X2 by Ry, and Ry, respectively. 


Note that the cross-covariance sequence is not symmetric about zero, that 
is, Ry2(v) # Ry2(—v), but rather 


Ri2(v) = Cov (Xi (t), Xo(t + v)) = Cov (Xo(t + v), X1(t)) = Roi(—v). 
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If we define the (2x2) matrix 


Ri1(v) Ry2(v) 


R(v) = 
4 Pas Ro2(v) 


) ve Z, 


we have that R(v) = Cov (X(t), X(t+ v)), which can thus be regarded as the 
autocovariance function of the bivariate series X. Note that R(v) = R7(—v) 
since R,2(v) = Roi(—v) and Ry2 and Ro» are symmetric about lag zero. Fi- 
nally, we can write 


R(v) = E ((X(t) — p)(X(t+v) —4)*), 
where p = E(X(t)) = (m1, 2)". 
The Cross-Correlation Function 
If we define 


Rij(v) 
V Ris (0) Rj5(0)’ 


then pi; and p22 are the usual autocorrelation functions of X 1 and X29, while 
P12 is called the cross-correlation function of X. If pi2(v) = Oforallv € Z, then 
we say that the series X, and X> are not cross-correlated. We will often just 
say that they are uncorrelated if there is no chance of confusion over whether 
we mean un-autocorrelated or not cross-correlated. 


pig (v) = Corr(X;(t), X;(t+ v)) = Vcr 47 1,2, 


Table 4.1. True Auto- and Cross-Correlations for a Bivariate Process 


v rhoi(v) rho2(v) rhoi2(v) rhoi2(-v) 


i) 1.000 1.000 - 038 038 
1 -583 517 -457 - .434 
74 -141 068 - 483 - .496 
3 -.136 ~.183 . 303 =.329 
4 =. 220 ~ .236 -092 -.114 
5 -.174 -.168 -.050 039 
6 -.081 - .067 =a 102 100 
Lf - .003 O11 - .086 091 
8 038 -045 ~-.044 049 
9 043 044 - .006 009 
10 028 026 016 -.015 
11 010 007 020 = .021 
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Cross-Correlations 


“1.0 
“12,0 -7.28 -2.4@ 2.40 7.20 12,08 


19.98 Bivariate Realization (Xi on top) 


0.0 12.0 24.00 36.0 48.08 60.08 


Figure 4.1. Plots of the Cross-Correlations Given in Table 4.1 and Two 
Realizations of Length 60 from the Process. 
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To illustrate the cross-correlation function of a bivariate time series, con- 
sider Table 4.1 where we have given the values of the true auto- and cross- 
correlations for a particular bivariate process. The cross-correlations are plot- 
ted in Figure 4.1 along with two realizations of length 60 from the process. In 
Example 4.1 we describe the macros that were used to produce this table and 
figure (see also Jenkins and Watts (1968), p. 332 for more information about 
this process). The cross-correlation of lag 0 is very small, indicating that the 
two series are practically uncorrelated simultaneously in time. The rest of the 
cross-correlations are approximately antisymmetric in nature; that is, they ap- 
pear to be symmetric about zero except that those for negative lags are almost 
the negative of the ones for positive lags. This is an unusual phenomenon but 
affords an interesting example of trying to interpret what cross-correlations 
mean. 


The fact that the correlations for lags 1, 2, and 3 are all large means that 
on the average the values of X, at time t and X> a little bit later (TZ. c0r's 
time units) are similar in size. On the other hand, the correlations for lags —1, 
—2, and —3 are also large but negative; that is, on the average the values of X, 
at time t and X2 a little bit earlier (again 1, 2, or 3 time units) are also similar 
in size, but on opposite sides of their respective means. In order to explain this, 
we consider the data plots. The univariate series tend to have what appear to 
be very crude “cycles.” These are very irregular and are of varying lengths and 
shapes. We will see the spectral densities of X, and X2 in Figure 4.2. These 
spectral densities have no sharp peaks but are consistent with the existence of 
the crude peaks and toughs in the data. If we look at the joint behavior of the 
cycles in the univariate series, we see that sometimes they are in phase and 
sometimes they are out of phase. In this context we consider two cycles to be 
“in phase” if peaks or troughs in the two series are almost aligned, while they 
are “out of phase” if a peak (or trough) in one series is almost aligned with a 
trough (or peak) in the other. The reason for the antisymmetric character of 
the cross-correlations is that when the cycles are in phase, X, tends to lead 
X2 slightly; that is, the peak or trough in Xj is followed by a similar peak or 
trough in X2 a few time units later (sometimes 1 time unit, sometimes 2 or 3): 
On the other hand, when they are out of phase, the peak (or trough) in X2 
tends to occur slightly before the trough (or peak) in X;. We will discuss this 
example further in this chapter (see Problem C4.1 also). See Example 4.1 for 
the macros that were used to form Figure 4.1. 


4.1.2. The Spectral Density Function 


We next consider whether there exists a harmonic analysis of the autoco- 
variance function R. We know from Section 2.1 that there are harmonic anal- 
yses for Ry; and R22. In our consideration of bivariate series we will assume 
that each of the three sequences Rj, R22, and Rj2 is absolutely summable, 
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and write this as a 
> [R(v)| < 00, 
vu=>-CcO 


and say that R is absolutely summable. Under this assumption we have the 
following theorem. Note that a matrix having complex elements is called a 
complex matrix, while the complex conjugate transpose of a complex matrix 
A is denoted by A* and has (i, j)th element given by the complex conjugate 
of the (j,7)th element of A. 


Theorem 4.1.1 | SPECTRAL REPRESENTATION OF R 


If the autocovariance function R of a covariance stationary bivariate time 
series X is absolutely summable, then 


a) There exists a (2x2) complex matrix function f, defined on [0,1], such 
that R and f are Fourier pairs; that is, 


le.e] 


f(w) = Se Ripjen 2, w € [0, 1] 


v=—0o 


1 
R(v) = fs f(w)e2™"™, veEZ. 
0 


The sum and integral are calculated for each element of the matrices involved; 
for example, we have 


fia(w) = 3 eee, 


uvu=-00 


b) The matrix f(w) is Hermitian, that is, f*(w) = f(w), f(w) is positive 
semidefinite, that 1s, 
h*f(w)h > 0 
for any nonzero two-dimensional complex vector h, and we have f(w) = f*(1— 
w). 


Proof: We prove part (b) only. Note that fi; and fo2 are the usual univariate 
spectral densities for X; and X2 and thus they are real and so are equal to 
their complex conjugates. We thus need only show that fi2(w) = foi(w) in 
order to show that f is Hermitian. We have 


fio(w ne Rio(v) 2" 


vu=-—0co 
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since (1) the complex conjugate of a sum is the sum of complex conjugates, 
(2) the complex conjugate of a real times a complex is the real times the 
complex conjugate of the complex, and (3) the complex conjugate of e~2"*¥ — 
cos 2rvw — isin 2mvw is cos 2myw + isin 2rvw = e?"*” Now if we let k = —v 
we get 


’ 


fio(w) = yi Rya(]k)eget he 


k=-0o 


= >. Rai (kyextn 
k=—00 


= fei(w), 


since Ri2(—k) = Ro, (k). 
_ _ Toshow that f(w) = f*(1—w), we again need only show that fi2(1—w) = 
foi(w) since fi; and f22 are symmetric about frequency .5. We have 


fio(1 =w) = DS Ri2(v)e~27*¥(1—-») 


vu=—o00 


2. . 
Ds Ryo(vje2™* 


v=-—00 


since e~?"*¥ — ]. But this last sum is clearly fo; (w). 


We next show that h?f(w)h > 0 for all nonzero real two-dimensional 
vectors h = (h;,h2)". If we let Y(t) = Ay Xi(t) + h2Xo(t), then it is easy 
to show (see Problem T4.1) that the univariate series Y has spectral density 
h7f(w)h. Recalling that the spectral density of a univariate time series is 
nonnegative gives the result. We can extend this to complex h, but this requires 
the introduction of complex valued time series which we prefer not to do. 


The spectral densities f;; and f22 are called autospectral densities, and 
the function f;2 the cross-spectral density of X. Before discussing the interpre- 
tation of the cross-spectral density function, we describe some functions that 
are derived from it. 


Functions Derived from the Cross-Spectral Density 
Since the cross-spectral density is complex valued, its value at a particular 
frequency contains two pieces of information, namely its real and imaginary 


’ parts. Thus we can write 


fi2(w) = c12(w) — igi2(w), 
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where 


C12(w) = Re(fi2(w)) and q12(w) = —Im(f12(w)) 


are called the cospectral density and quadrature spectral density, respectively. 
We will see presently why q12 is defined to be the negative of the imaginary part 
of the cross-spectral density. We can also express the cross-spectral density in 
terms of its amplitude and phase (see Problem T1.2) by 


fi2(w) = A1a(w)e*e2) | 


where 


Ai2(w) = |fi2(w)| and $12(w) = arctan a 


are called the amplitude spectrum and phase spectrum, respectively. See Prob- 
lem T1.2 for a discussion of the arctangent function. 


The POLAR Command 


If the arrays zr and zi contain the real and imaginary parts a; and 6; of 
the n complex numbers 21,..., Zn, then the command 


POLAR(zr,zi,n,amp, phase) 


will return A; = |z;| and ¢; = arctan(b;/a;) in the arrays amp and phase of 
length n. 


We can also define various standardized forms of the cross-spectral density, 
including the complex coherency function 


wi2(w) = pli 1S BE 
fir (w) fo2(w) 


the coherency function 


Wi2 (w) = |wi2(w) |, 


and the squared coherency function W?,(w). Sometimes we will refer to the 
coherency, amplitude, and phase spectral densities as the coherence spectrum, 
amplitude spectrum, and phase spectrum, respectively. We will see that the 
squared coherency function plays the role of a multiple correlation coefficient. 
The next theorem shows that it lies between zero and one. 
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Theorem 4.1.2 | RANGE OF COHERENCE SPECTRUM 


If both autospectral densities are positive, then the coherency function 
and thus the squared coherency function are between zero and one for all 
frequencies. 


Proof: The fact that |w;2| is nonnegative follows from the fact that it is the 
ratio of nonnegative functions. The fact that |wi2(w)| < 1 follows from the 
fact that 


fir (w) fo2(w) — fr2(w) foi (w) > 0, 


which is true since (1) this is the determinant of f(w) which is positive semidefi- 
nite by part (b) of Theorem 4.1.1, and (2) the determinant of a positive semidef- 
inite matrix is nonnegative. 


Later we will estimate the functions defined above by substituting esti- 
mates of the auto- and cross-spectral densities into their formulas. Thus we 
will use only autospectral estimates that are guaranteed to be positive. Thus 
if one uses the truncated periodogram or Tukey windows to estimate the au- 
tospectra, the coherence need not be between zero and one. To illustrate the 
appearance of the squared coherence and phase spectra of a bivariate time 
series, consider Figure 4.2, which contains graphs of the autospectra, squared 
coherence, and phase spectra for the process in Figure 4.1. In Example 4.1 we 
describe how the auto- and cross-spectra were calculated for this series. The 
following macro can be used to form plots of the squared coherence and phase 
spectra for any process given (true or estimated) auto- and cross-spectra. 


53 CROSSP.MAC: macro to find and plot the phase and squared 

He coherence functions given autospectra f11 and £22 
oi and the real and imaginary parts fi2r, f12i of the 
ne cross~spectra of a time series. 

a INPUT: Q (the number of frequencies between O and 1) 

4 fil £22, for. F121 

PAUSE 

jstart 

q=Q/2 

q=qt1 

POLAR(f12r,f12i,q,amp,phase) ;POLAR gives arctan(b/a) not arctan(-b/a) 
coher=amp*amp 

coher=coher/fii 

coher=coher/f22 

LABEL(coher)=’Squared Coherency Spectrum’ 

LABEL (phase)=’Phase Spectrum’ 

aa=<0,1> 


a 
FOoMP MONA Ah wWN PR 


[Sl el 9 
oOo WN AN Hh WwW 
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Autospectra of First Series Squared Coherence Spectrum 


6.0 1.0 

3.68 0.80 

1.28 0.68 

-1.20 0.48 

-3.60 0.26 

-o,. é. 
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3.68 1.0 

1.20 9.63 

-1.20 -,63 
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=< “3.14 

a @1 02 03 @4 @5 00 @1 62 @3 0.4 0.5 


Figure 4.2. The Autospectra, Squared Coherence, and Phase Spectrum for 
the Process in Figure 4.1. 


ry 


21 freq=POLY(aa,1,q,0,.5) ;find frequencies 
22 LABEL(freq)=’ ’ 

23 PLOT(freq,coher,q,0,.5,0,1) 

24 mpi=-pi 

25 PLOT(freq,phase,q,0,.5,mpi,pi) 


on 


Note that the autospectra in Figure 4.2 have some power in frequencies 
corresponding to the cycle lengths in the data sets in Figure 4.1. The phase 
spectrum is roughly constant and the squared coherence is large for moderate 
frequencies and small in its tails. We will discuss the interpretation of these 
functions next. 


Interpreting Cross-Spectra 


In Sections 2.1 and A.2, we interpreted the autospectral density at a fre- 
quency w as the average value of the amplitudes of the frequency component 
of frequency w for many realizations of a univariate time series. To interpret 
cross-spectra, we must visualize generating many long realizations of the bi- 
variate series, and for each bivariate realization, calculating the coefficients of 
the cosine and sine terms in the sinusoidal decomposition of both univariate 
realizations. Thus for each bivariate realization, we will have four numbers for 
each frequency. Denote the cosine and sine coefficients for the frequency w for 
a univariate realization by a;(w) and b;(w) where i is 1 for X, and 2 for X9. 
Then we have the following interpretation for the quantities derived from the 
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cross-spectral density. 
Cospectrum and Quadrature Spectrum 


The cospectrum ci2(w) is the sum of the covariance of a,(w) with a2(w) 
and the covariance of 6;(w) with b2(w). On the other hand, the quadrature 
spectrum 412(w) is the sum of the covariance of a;(w) with b2(w) and the 
covariance of a2(w) with b;(w). Note that the quadrature spectrum is defined 
to be the negative of the imaginary part of the cross-spectrum so that we 
do get this covariance interpretation. Thus the cospectrum for frequency w 
measures the strength of the linear relationship between the coefficients over 
many realizations of like trigonometric functions (cosine or sine) for frequency 
w, while the quadrature spectrum is for opposite trig functions. This is the 
origin of the prefix “co” and “quadrature.” 


Amplitude and Phase Spectra 


The amplitude spectrum Aj2(w) combines the information about the co- 
efficients contained in the co- and quadrature spectra. Thus it measures the 
relationship between the amplitudes of the sinusoids in the univariate realiza- 
tions at frequency w. The phase spectrum ¢12(w) measures how out of phase 
the frequency components for the univariate realizations tend to be. The phase 
is a number between —z and 7, with —7 or 7 indicating that the components 
are perfectly out of phase. For example, if ¢12(1/12) = 7, then the frequency 
components of period 12 tend to be perfectly out of phase; that is, if the data 
are monthly, then the 12-month frequency components are shifted 6 months 
from each other. If the autospectra indicate that the 12-month components are 
predominant, then ¢12(1/12) = m would indicate that the two series are rather 
cyclic (of period 12 months) but perfectly out of phase, with one series leading 
(or lagging) the other by 6 months. If, on the other hand, ¢12(1/12) = x/2, 
then one series leads (or lags) the other by 3 months. We note that the phase 
does not indicate which series is leading the other. 


The Coherence and Squared Coherence 


The coherency spectrum is merely a normalized version of the amplitude 
spectrum, that is, 


Aj2(w) 


Pa) i V fir(w) fo2(w) 


If we recall the spectral representation of a univariate series, then we could 
show that the coherency spectrum at frequency w is the absolute value of the 
correlation coefficient of the random amplitudes of the sinusoids of frequency 
w in the two univariate series. Further, the squared coherence at frequency w 
plays the role of the multiple correlation coefficient R? if we were to regress the 
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amplitudes of the frequency components of frequency w in one series on the 
corresponding amplitudes for the other series over many bivariate realizations. 


An Example of Interpreting Coherence 


To illustrate the repeated realizations interpretation of the coherence, con- 
sider Figure 4.3 (see Example 4.2). In this figure, we generated 100 realizations 
each of length n = 200 from the bivariate time series that we discussed in Fig- 
ures 4.1 and 4.2. For each bivariate realization we calculated the periodogram 
of the individual univariate realizations at Q = 200 frequencies between 0 and 
1. We then extracted the pair of periodogram ordinates at frequencies .025, 
.125, and .40. Thus at the end of this experiment we had three sets of 100 pairs 
of amplitudes. The scatterplots of these three data sets are included in Fig- 
ure 4.3. Note that the true coherence spectrum is large in the region around 
frequency .125 and small near frequencies 0 and .5. This is reflected in the 
three scatterplots. Those for frequencies .025 and .4 show no linear pattern, 
while the one for frequency .125 has evidence of strong correlation. Note that 
these scatterplots are somewhat unusual in appearance because they are for 
x? variables since the periodogram has a x? distribution. They seem unusual 
because we are used to looking at scatterplots of normally distributed random 
variables. Note also that the values of the autospectra (see Figure 4.2) are 
large in the region of frequency .125 and small near frequencies 0 and .5. This 
is also reflected in the scatterplots. 


Bivariate White Noise 


We next turn to the bivariate analog of a white noise time series. It is clear 
that we would like the univariate series X; and X» to be white noise, and that 
X, and X» should be uncorrelated at different times. The traditional definition 
of bivariate white noise processes allows there to be nonzero correlation between 
X, and X» at the same time point. 


Definition. A bivariate time series X is said to be a white noise series if 
E(X(t)) = 0, while 
ee 0 
R(v) = 


0, v#0 


for some positive semidefinite matrix 


For such a process, we write X ~ RW(Z). 
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Figure 4.3. Interpreting the Coherence Spectrum of a Bivariate Time Series. 
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For a white noise series, we have 


fi2(w) = o12 


_ |or2| 
|wi2()| = Rize 


$12(w) = 0. 


Thus the cross-spectral density is also a constant, while the coherence is a con- 
stant equal to the simultaneous correlation between X, and X2 and the phase 
spectrum is identically zero. Thus on the average, the frequency components 
in the two series are in phase. 


Two other important uses of the coherency spectrum are described in the 
next theorem. 


Theorem 4.1.3 | TWo PROPERTIES OF COHERENCE 


Let X be a covariance stationary bivariate time series with autocovariance 
function Rand spectral density function f. Then 


a) The univariate series X; and X> are not cross-correlated if and only if 
the coherency spectrum is zero for all frequencies. 


b) If Y; and Y2 are filtered versions of X; and X2, then the coherency 
spectrum of Y is the same as that of X; that is, coherency is unaffected by 
linear transformations. 


Proof: Part (a) follows immediately from the fact that 


1 
Ry2(v) =) fio(w)e?"* dw, 
0 


and the Fourier coefficients of a function are all zero if and only if the function 
is identically zero. In Problem T4.2 we discuss the proof of part (b), which 
follows from the bivariate Filter Theorem which we consider next. 
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4.1.3. Bivariate Filters 


In Section 2.3 we discussed determining the properties of the output of 
a filter from that of the input. We now extend these results to include the 
calculation of the cross-covariances and cross-spectra between the input and 
the output. We also consider the more general case where a bivariate series Y 
is a filtered version of the bivariate series X. 


Definition. The bivariate time series Y is said to be a filtered version of 
the bivariate series X with filter coefficient matrices {B(j),j € Z} and filter 
operator G(L) = )>*°_. B(j)L’ if 


'j=—0o 


¥(t)= 5° B(j)X(t- 5) = G(L)X(t) 


j=—oo 


exists as a limit in mean square. 


An example of this bivariate filter is the bivariate moving average process 
X defined by 


X(t) = > B(je(t— 8), 
k=0 


where € is a bivariate white noise series. 


Theorem 4.1.4 | BIVARIATE FILTER THEOREM 


a) If the univariate time series X> is a filtered version of the univariate 
series X,, that is, 


X(t) = »s B;Xi(t — Jj), 


then 


1) The cross-covariance function Rj» satisfies 


Rio(v)= > BRulv—j), vez. 


j=-o 


li) The cross-spectral density function fy» is given by 


fi2(w) = g(e~?"*”) far (w), 
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co 
where g(z) = ye Bye. 
j=—oo 


b) If the bivariate series Y is a filtered version of the covariance stationary 
bivariate series X, then Y is also covariance stationary and the autocovariance 
and spectral density functions Ry and fy of Y are related to those of X by 


Ry()= 32  BY)RxG +¥— HB) 


j=—0o k=—00 


fy (w) = G(e?"”)fx (w)G* Coed : 


Proof of part (b): For convenience we assume that the mean vector for X 
is the vector of zeros. Thus we have 


Ry (v) = E (X(t)X7 (t+ »)) 


co co Yi 
EAE Oe aux ( DE Bext-+»-m) 
k 


j=-~ 


3 53 B(j)E (X(t — j)X7 (t+ v — k)) B7(k) 


j=—@~ k=-0co 


co co 


i 
M 


B(j)Rx(j + uv — k)B7(k). 


j=—-©% k=-0o 


From this expression one can use some tedious algebra to verify the expression 
for fy. 


We note the similarity of the expressions in this theorem to those in the 
univariate case (see Problem T4.3). The difficulty in the bivariate case is that 
the matrices involved do not commute. Thus fy is the product of three matrices 
and we cannot alter the order in which they are multiplied. 


4.1.4. Filters and Cross-Spectral Analysis 


The use of the cross-spectral density function to study a bivariate time 
series is called cross-spectral analysis. In this section we consider using cross- 
spectral analysis in two ways. First we use it to describe the effects of a 
filter being applied to a univariate series. Then we will see how cross-spectral 
analysis can be employed to determine whether one univariate series can be 
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thought of as a filtered version of another. This is the bivariate time series 
analog of simple linear regression. Note that we will be assuming that we 
know the quantities involved. In Section 4.2 we discuss using these methods 
when we must estimate the cross-spectra. 


Describing the Effects of a Filter 


Suppose that the univariate series X> is a filtered version of the univariate 
series X1; that is, suppose that 


X2(t)= $7 BjXi(t- 3) = g(L)Xi(2). 


j=—00 
Then we have by the bivariate Filter Theorem that 
firo(w) = g(e-"™) firw) and fo2(w) = |9(e?"”) [? fr(w), 
which means that the coherence spectrum of X, and Xp is given by 


Wi2(w) = lfi2(w)| 


V fir () fo2(w) 


___ lye?) fix(w) 
Vial)loe™)P fia) 


= w € [0,1], 


which means that the amplitudes of the frequency components for a specified 
frequency in the input and output series are perfectly correlated, and this is 
true for every frequency (see Problem C4.2). Now two random variables (such 
as these two amplitudes) are perfectly correlated if and only if one is a linear 
function of the other. To get an idea of what the filter does to the amplitude of 
the frequency component of frequency w in the input series, we recall that the 
spectral representation of a univariate time series allows us to think of such a 
series as being made up of the sum of many sinusoids and that these sinusoids 
can be considered separately. Thus we find what the filter would do to a series 
consisting of just a sinusoid of frequency w, that is, to X; (t) = cos 2mtw. We 
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have 


Xa(t)= D7 Bj Xilt-j)= YY Bjcos2m(t — j)w 


j=-0oo j=-0 


co 
= bes B; [cos 2ntw cos 2mjw + sin 2rtw sin 2rjw] 


j=-00 


co co 
cos 27tw ye 8; cos 2mjw| + sin 2rtw ye 8; sin 2ajw 


j=-oo j=-© 


= cos 2rtwRe [g(e?"”)] + sin 2rtwIm [g(e?"*”)| 


|g(e?"*”)| cos (27tw + ¢(w)), 


where : 
Re (ger) 
Im (g(e?"*)) © 


Thus the output of the filter is also a sinusoid of frequency w, except (1) it has 
been multiplied by the modulus of the frequency transfer function of the filter 
(which is also known as the gain of the filter), and (2) it has been shifted out 
of phase with the input sinusoid by the amount determined by ¢ (see Problem 
T1.3 for a review of phase shifts). In analogy with terminology in electronics, 
we call the function 

_ Ifi2(w)| 


92\1(w) a fulw) 


the gain spectrum of X> given X,. Thus if X2 is in fact a filtered version of 
Xj, then the gain and phase spectra will be the same as the gain and phase of 
the filter. 


¢(w) = arctan 


The Phase Spectrum of a Delay Process 


One important relationship between univariate series X,; and X» is the 
case where 


X(t) = BXy(t — d) + e(t) 


for some lag d, where € is a white noise process having variance o? and is 
uncorrelated with X,. Thus X2(t) is a noisy and time delayed version of X. 
For convenience we assume that X, has zero mean. Thus we have 


Ryo(v) = E[Xi(t) (Xi(t + v — d) + e(t + v))] 


— Rii(v = d), 
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and thus 


fi2(w) = Dig Ru(v — d)e~?™iv 


vu=-—00 


oe) 
— e~2midw ys Ry (v i d)e~2"i(v~d)w 


v=>-—o0o 
= ene fy (@) 
= cos Imdwf11(w) — isin 2mdw fy1(w). 


This means that the phase spectrum is 


— sin 27dw 


= arctan ————=——— 
Pr2(w) Ser cos 27rdw 


= arctan (- tan 2ndw) 
= —2rdw; 
that is, the phase spectrum is a linear function of w with slope given by 2rd. 
Determining an Optimal Filter 
Given two univariate series X; and X2, we now seek to find the filter 


{83,7 € Z} such that 


2 


E | X2(t)— D> B;Xi(t— J) 


j=-0 


is as small as possible; that is, we seek the filter that when applied to X, results 
in the series that is as close to X2 (in the mean square error sense) as possible. 
The solution to this problem is provided in the next theorem. 


Theorem 4.1.5 | OPTIMAL FILTERS 


Let X(t) = (Xi(t), X2(t))? be a covariance stationary bivariate time series 
having spectral density f. Then 


a) The filter that when applied to X, best approximates Xz in the mean 
square error sense is given by 
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b) The residual time series 
dt) = Xl) - > Xt f 
j=-oo 


has spectral density function 


fe(w) = fo2(w) — for(w) fq’ (w) fa2(w) 


|fi2(w)/? 
fiw) 


= foa(w) [1- Wi,()], 


= fo2(w) aay 


which means that the filtered version of X, perfectly matches X> for the fre- 
quency component of frequency w if and only if the squared coherence W}?, is 
one at frequency w. 


Implications: This theorem essentially provides a regression analysis of one 
univariate series on another, except that there is a regression for each fre- 
quency, with the squared coherence playing the role of R? at each frequency. 
Note further the similarity of the form of the functions involved to those in 
ordinary regression analysis if we think of X as the “input” to the regression 
model and y as the “output.” For example, the filter coefficients are given 
as the Fourier transform of the function f12/f11 (which is called the regres- 
sion transfer function), while least squares regression coefficients are given by 
B = (X7X)-!X7y, which is of a similar form with X7 X and X7y playing the 
roles of fj; and f,2. We also have that the residual sum of squares in regression 
is of the form RSS = y7y — y7(X7X)~'y, which is analogous to the formula 
for the spectral density of the error time series. 


Calculating Filter Coefficients 


In practice, we will only know the values of f;2 and f1; (or estimates of 
them) at the frequencies w; = (j — 1)/Q, j = 1,...,Q, for some integer Q. 


Thus we will approximate the integral in part (a) of Theorem 4.1.5 by the 
rectangular sum 


sy ees fi2(we) (wk) e2mijwe 
fir (wx) 


We have the following results for this approximation. 
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Theorem 4.1.6 | APPROXIMATING FILTER COEFFICIENTS 


Let B; be the rectangular sum approximation (based on Q frequencies) to 
the filter coefficient B;. Then 


a) For j > 0, B; is the real part of the (j + 1)st element of the discrete 
Fourier transform of 
fi2(w1) fi2(wa) 


fire.) far(wg) 


b) B-3 = Ba-3. 


c) If B; =0 for |j| > Q, then B; = p;. 


Thus to find the coefficients of an optimal filter, we will (1) evaluate (or 
estimate) the auto- and cross-spectra, and then (2) use the FFT command to ap- 
proximate the integrals involved. See Problem T4.6 for a further simplification 
of part (a). 


The Double Command 


As in the univariate case, we will usually calculate spectral functions at 
the frequencies (j —-1)/Q for j = 1,...,Q, but store only the first q = [Q/2]+1 
values. To use the FFT command to do the rectangular sums approximation, 
we will need the spectral arrays at all Q frequencies. Thus given the real and 
imaginary parts of f,2 at q = [Q/2]+1 frequencies in the arrays f12r and £423. 
the command 


DOUBLE(f12r,£12i,Q) 


extends the arrays f12r and f12i to length Q by using the facts that the real 
part is symmetric about 0.5, while the imaginary part is antisymmetric about 
0.5; that is, the values at frequencies w and 1 —w are the same except they 
have opposite signs. z 

The following macro will calculate the §;’s corresponding to arrays f12r, 
£12i, and f11. 


oy FINDFILT.MAC: This macro accepts the cross-spectra 

is fxyr and fxyi of two series and the 

OS autospectra of the first in fxx, and then 
approximates the integrals in the Fourier 
He transform to find the best fitting filter 
- coefficients. 


CANA Hh WYN 


oF INPUT: fxyr, fxyi, fxx (cross-spectra and autospectra at 
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10: 5; q=[Q/2]+1 points) 

Th Tepe te Q (number of frequencies between O and 1 at which spectra 
NP) es were calculated). 

The m (number of coefficients to find) 

14 ;; 

15 PAUSE 

16 ;start 

17 fxyri=fxyr/fxx ;divide by fxx 

18 fxyil=fxyi/fxx 

19 DOUBLE(fxyri,fxyii,Q) ;double arrays by symmetry 

20 mpi=m+1 

21 FFT(fxyri,fxyii,Q,mpi,1,fxyri,fxyil) 

22 betaO=fxyri[i]/Q ;pick off first element 
23 beta=EXTRACT(fxyr1,2,mp1) ;get the next m 


24 beta=beta/Q 

25 LABEL(beta)=’Filter Coefficients’ 
26 LIST(beta0) 

27 LIST(beta) 


As a simple example, suppose 


where Xj, is white noise with variance 1. Thus (see Problem T4.7) 


gli> det) 


The next macro calculates this function at a user-specified number of frequen- 
cies and then calls the FINDFILT macro to approximate the filter coefficients. 


Lagis 

poe FILTEX.MAC: This macro finds the transfer function of the 

a 3% filter 

AP sis 

5 33 y(t)= 1/2 [ Sum(j=0,infinity) (1/2)7j x(t-j)] 
Ges 

(ene and then calls FINDFILT to find the coefficients. 
8 55 

ie INPUT: Q (number of frequencies between O and 1) 

10 ;; m (number of coefficients to find) 

ba Py gh 

12 PAUSE 

13 ;start 

14 q={Q/2}+1 

15 freq=LIWE(q,0,.5,1) ;freq contains [Q/2]+1 frequencies from 0 to .5 


i 
nD 


freq=2*pi*freq 
cc=C0S(freq,q) 


e 
x 
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18 ss=SIN(freq,q) 

19 fxyr=1-.5*cc 

20 fxyi={-.5}+*ss 

21 d=2*{fxyr”2+fxyi~2} 
22 fxyr=fxyr/d 

23 fxyi=fxyi/d 

24 fxx=LIWE(q,1,0) 

25 fyy=LIWE(q,1,0) 

26 MACRO(findfilt start) 


If we use this macro for Q = 10, then the 8’s are determined correctly to 
five decimal places. Note that the amplitude of this transfer function is very 
smooth and thus the approximation is quite good even for small Q. In cases 
where the cross-spectrum is not so smooth, we will have to use larger values of 


Q. 
4.1.5. The Bivariate Autoregressive Process 


As in the univariate case, the bivariate autoregressive process is very im- 
portant in modeling data. A time series X is said to be an autoregressive 
process of order p with (2x2) coefficient matrices A(1),...,A(p) and error 
covariance matrix © if the process satisfies the stochastic difference equation 


P 
YOAW)X(t-j)=e(t), te Z, 


j=0 


where € is a bivariate white noise time series with covariance matrix ©. Note 
that this equality must be interpreted in a fashion similar to the interpretation 
for the univariate case. Thus we define the autoregressive operator 


P 
G(L) = AGLI. 
j=0 
If we can invert G in terms of nonnegative powers of L only, that is, as 
co 
-4(L) =)? B(k)L*, 
k=0 
then we have that = 
X(t) = )> B(ke(t — 3) 
k=0 
exists as a limit in mean square and satisfies the AR difference equation. To 


obtain the B’s, we equate like powers of z in the equation G(L)G~1(L) = I, 
and obtain B(0) = Iz and 


min(p,j) 


Bij)=- > A()BG-i), j>1. 


s=1 
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In the univariate case, the AR operator can be inverted in terms of nonnegative 
powers if and only if the zeros of the associated complex polynomial g(z) = 
woe a;z? are all outside the unit circle. The analogous condition in the 
bivariate case is that the zeros of the determinant of the matrix of polynomials 
G(z) = x —) A(j)z/ are all outside the unit circle. We state this as a theorem. 


Theorem 4.1.7 | STATIONARITY OF AR 


The stochastic difference equation defining a bivariate AR process has a 
stationary solution in terms of present and past e’s if and only if the zeros of 


WA 


j=0 


are all outside the unit circle. 


To illustrate this theorem, consider p = 1 and denote the single coefficient 


matrix by 
411 412 
421 422 


P 
So AG) = Ile + Ad 
=0 


Then we have 


1+ a41z 422 


a21z 1+ ag2z 
= 1+ (a1; + G22) z + (@11422 — a12421) 
= 1+4 tr(A)z + |A|z?. 


Note that for general p, the determinental polynomial will be of degree 2p 
and the coefficients will not be so easily determined as in this example (see 
Problem T4.8). In the first-order case, we can operate as though the second- 
degree polynomial is that of a univariate second-order process with coefficients 


a,=tr(A) and a2=|Al. 


This univariate process has partial autocorrelations (see Theorem 2.6.4) 02 = 
—ay and 6; = —a;/(1+ a2). Thus the bivariate process is stationary if and 
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only if (1) the absolute value of the determinant of A is less than 1, and (2) 
the ratio of the trace of A to 1 plus its determinant is less than 1 in absolute 
value. 


The bivariate time series that we described in Section 4.1 (see Figures 4.1 
and 4.2) is the autoregressive process having coefficient matrix A and error 
covariance matrix © given by 


—0.6 0.5 1.0 0.0 
A= and = ; 
—0.4 —0.5 0.0 1.0 


Thus if we let €; and €, be two independent univariate Gaussian white noise 
time series, each having variance 1, we have 


Xj (t) = 0.6X1(t = 1) — 0.5Xo(t _ 1) + €1(t) 
X2(t) = 0.4X,(t —_ 1) + 0.5Xo(t = 1) + €2(t). 


Note that the coefficients have changed signs from those in the matrix A be- 
cause we have put X;(¢— 1) and X2(¢ — 1) on the right side of the equal sign. 
This model says that X, at time t is approximately the difference of X 1 and 
X2 at the previous time (plus noise), while X> is the almost the average of 
X and X» at the previous time (also plus noise). The cross-correlation and 
cross-spectral density functions summarize the joint behavior of the two series. 


The ARCORR2 Command 


If the zeros of |G(z)| are all outside of the unit circle, then we have 


co 


X(t) = S> B(k)e(t — k), 


k=0 
which means by the bivariate Filter Theorem that 
co 
R(v) = Cov(X(t),X(t+v)) = }> B(k)EB? (k + v). 
k=0 


The ARCORR2 command uses this series to find the autocovariances. If it does 
not converge within a reasonable time (ARCORR2 will use as many as 100 terms 
in the sum), then ARCORR2 issues a message that the process is not stationary 
and aborts. The command is of the form 


ARCORR2(A,p,SIGMA,M,R10,R20,rho0120 ,rhoi,rho2,rho12,rho21, ier) 


where M is the number of lags at which correlations are wanted. The output con- 
sists of the variances R10 and R20 of the univariate series, the cross-correlation 
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of lag zero in rho120, the first M autocorrelations of the univariate series in the 
arrays rho1 and rho2, and finally the cross-correlations p;2(k) fork =1,...,M 
in the array rhoi2 and po;(k) for k = 1,...,M in the array rho21. Recall that 
p21(v) = pi2(—v), and thus rho12 (rho21) contains cross-correlations between 
X(t) and X2 at later (earlier) times. If the process is judged to be nonstation- 
ary, the output integer variable ier is set to 1 and no other output is returned. 
If the process is judged to be stationary, ier is set to 0. See Example 4.1 for 
an illustration of the use of the ARCORR2 command. 


The ARDT2 Command 


As in the univariate case (see the ARDT command), TIMESLAB has two 
forms of the command for simulating bivariate AR processes: 


X=ARDT2(A,p,n,e,ier) 
and 
X=ARDT2(A,p,sigma,seed,n,ier). 


The first form actually finds the future terms of a general bivariate difference 
equation 


Pp 
X(t) =e(t)—S°AG)X(t-3j), t=ptl,...,n, 
g=1 


where for t = 1,...,p, X(t) is set equal to the starting values e(t). The A’s 
and e’s are entered in the arrays A and e, while the order p and length n are 
entered in the integers p and n. The output integer ier is 1 if any term in 
the difference equation becomes larger than 101°. As in the univariate case, if 
the zeros of the determinental polynomial are not outside the unit circle, the 
terms of the sum can become explosive, and thus TIMESLAB guards against 
this possibility. If it happens, then no X is returned. If the terms are not 
explosive, then ier is set equal to 0. When using this command it is important 
to remember that matrices (such as e and X) are stored in TIMESLAB by 
column, while three-dimensional arrays (such as A) are stored first by column 
and then by index. For example if we have a second-order process with 


a b eae? 
A(1) = and A(2)= ; 
fe fel g h 
then the array A should be of the form 


A=<a,c,b,d,e,g,f,h>. 
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The second form of the ARDT2 command generates a realization having 
Gaussian errors, and has the additional input arguments sigma and seed con- 
taining the desired error covariance matrix and random number generator seed. 
The output integer ier has the same meaning as in the first form of the com- 
mand. Note that the second form of the ARDT2 command was used to generate 
the data in Figure 4.1 (see Example 4.1). 


The ARSP2 Command 


From the bivariate Filter Theorem we have that the spectral density func- 
tion of the white noise error series is given by 


f.(w) = = G(e?"* fx (w) G* (e274), 


and thus since the determinant of G(e?"™) is never zero for w € [0, 1], we can 
invert it (and its complex conjugate transpose) to obtain 


fx (w) = Gt (et) nG-* (e278); 


where by G~* we mean the inverse of the complex conjugate of the matrix G. 
The command 


ARSP2(A,p,sigma,q,f11,f22,f12r,f12i) 


returns the autospectra f,; and f22 in the arrays f11 and £22 and the real and 
imaginary parts of the cross-spectra f12 in the arrays f12r and £12i, all at the 
frequencies (j — 1)/q for j = 1,...,[q/2]+1, where q is entered in the integer 
q. The FFT algorithm is not used in this command, so q need not satisfy 
any of the rules for number of frequencies as in the univariate case. Again see 
Example 4.1 for an illustration of the use of ARSP2. 


The Bivariate Yule-Walker Equations and Levinson Algorithm 


We next turn to the bivariate analog of the Yule-Walker equations. If 
we multiply the bivariate AR stochastic difference equation on the right by 
X7(t — v) and take expected values of both sides of the result, we obtain 


AYRE “a v) = 6,%, v>20, 


j=0 


where 6, is the Kronecker delta function; that is, d, is zero unless v = 0 in 
which case it is one. This is again because X(t — v) is uncorrelated with e(t) 
for positive v, while 


Cov (X(t), e(t)) =X. 
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Thus we again have a system of linear equations (called the bivariate Yule- 
Walker equations) relating the coefficients and autocovariances of a bivariate 
AR process. If we know the autocovariances R(0), R(1),...,R(p), we can 
find the corresponding coefficients and error covariance matrix by the bivariate 
Levinson algorithm given in the next theorem. This algorithm is due to Whittle 
(1963b). 


Theorem 4.1.8 | BIVARIATE LEVINSON ALGORITHM 


Suppose that X is a bivariate AR process of order p with coefficients 
A(1),...,A(p) and noise variance &. For k = 1,...,p, let the matrices A;(j), 
Bx(j),j =1,...,k, and XZ; andT,, be the solutions to the systems of equations 


k k 

Do Ax(D)R(U- 7) = 62, >> Be(JRG-v)=4&0, 0 =0,...,k. 
4=0 3=0 
Then 


a) These matrices can be found recursively by defining Uo = To = R(0) 
and Ap = R™(1), @) = R(1), and then fork = 1,...,p: 


Ax(k) =—Ag-alz),,  Be(k) = —,-1E;2,, 


Ax (j) = Ax-i(j) + Ax (k)Be-i(k—j),  j=1,...,k-1 


B.(j) = Be-1(j) + Be(k)An-1(kK—-J), g=1,...,k-1 


k k 
Be = DAKG)RG), Th = Be (J)R7(3) 
k k 
An =O AR()R™(k+1-J), Se =) BAG)R(R+1-5). 


b) The coefficients of the AR process are given by A,(j),j = 0,...,p, 
while & = &,. 


c) The coefficients and prediction error covariance matrix of the best linear 
unbiased predictor of X(t) given the previous k X’s are given by the A;,’s and 
I',, while those for X(t) given the next k X’s are given by the B,’s and ®,. 


In the univariate case, the forward and backward prediction coefficients in 
Levinson’s algorithm were the same (except in reverse order) and the algorithm 
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greatly simplified. Note that any coefficient matrix having index zero is the 
identity matrix. Also, when k = 1, we skip the steps for A,(j) and B,(j) 
while when k = p, we don’t calculate A; or ®,. 


’ 


The CORRAR2 Command 


As in the univariate case, TIMESLAB has a command that can be used 
to find AR parameters from autocovariances. This command is called CORRAR2 
and the input can be either true or estimated correlations. When using esti- 
mated correlations, the user can tell CORRAR2 to determine the best order to use 
by using a separate form of the command (see Section 4.2.3). The command 
for known order p is of the form 


A=CORRAR2(R10,R20,rho120,rho1 »rho2,rho12,rho21,p, sigma,ier) 


where the variances of the univariate series are entered in the real scalars 
R10 and R20, the cross-correlation of lag 0 is entered in rhoi20, while the 
autocorrelations for the two series and the cross-correlations of positive and 
negative lags are entered in the p-dimensional arrays rhoi1, rho2, rhoi2, and 
rho21, respectively. Finally, the input integer p contains the order of the AR 
process, while the AR coefficients and error covariance matrix are returned in 
the arrays A and sigma respectively. The output integer ier is 1 if a singular 
matrix is encountered in Levinson’s algorithm, and 0 otherwise. If ier is il 
then A and sigma are not returned. 

Because there are so many arguments, great care must be taken to avoid 
exceeding the limit of 72 characters for a command. An illustration of the use 
of CORRAR2 is included in Section 4.2.3. 


4.1.6. The Bivariate ARMA Process 


The bivariate analog of the ARMA process is defined as the solution to 
the stochastic difference equation 


X(t) + A(1)X(t-1)+---+A(p) = €(t) + B(1)e(t — 1) + --- + B(q)e(t — q), 


which we can also write in terms of the AR and MA operators 


G(L) = AGL! and H(L) = 32 B(A)L! 
k=0 


j=0 


as G(L)X(t) = H(L)e(t). The process is stationary in terms of present and 
past ¢’s if and only if the zeros of the determinant of G(z) are all outside the 
unit circle, and is invertible if and only if the zeros of the determinant of H(z) 
are all outside the unit circle. The results that we described for univariate 
ARMA processes have analogs in the bivariate case (see Brockwell and Davis 
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(1987), p. 407 for some of them), but in this book we will primarily consider 
the bivariate AR model. 


4.1.7. The Bivariate General Linear Model 


In Chapter 3 we used the univariate general linear model 
co 
X(t)= S> Bee(t-k), © ~ WN, 
k=-—00 


as a general model for stating theorems about descriptive statistics. In this 
chapter we will use the bivariate analog 


X()= S> Bet — 2, 


k=-—00 


where € is a bivariate white noise series. 
4.2. Statistical Inferences for Bivariate Series 


In this section we consider the bivariate versions of the inferences of Chap- 
ter 3. We begin by considering descriptive statistics. 


4.2.1. Descriptive Statistics 


Given data X(1),...,X(n) from a bivariate time series X, we can define 
the sample mean, autocovariance, and spectral density functions by 


X= — S> X(t) 
t=1) 
n—|v| 
Ro) = — S> (X() — KK + ol) -X)?, fol <n 
t=1 
f(w) = p Rivjey 7424) aortas ] (01) 
v=—(n-1) 
Note that Fs (v) i (v) 
R(v) - 11\U ¥e 
Roi(v) Ro2(v) 
where 
. 1 n—|v| 


Ris(v) = = DO (Kilt) — KI (X(t+ we) - XH), bel <n, 


x | 
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while we can show (see Problem T4.9) that 


f(w) = ~ (x: (X(t) — X) sie) (x: (X(s) — X) aa : 


This last equality is important for showing an important property of the bi- 
variate sample spectral density, namely that at each frequency it is a singular 
matrix. We have that f (w) is the product of a vector times its complex conju- 
gate transpose. The rank of a product of matrices is less than or equal to the 
smaller of the ranks of the two matrices. In this case the matrices are vectors 
(which have rank at most 1) and thus f(w) is of rank at most 1. Thus its 
determinant must be 0. But 


F(w)| = fir(w) fo2(w) ar fi2(w) far (w), 


which means that fir(w) fro(w) = fio(w) fuw), and the sample coherence 
spectrum 


W42(w) = free) Sle We [0, iN 
fir (w) fo2(w) 


which renders it useless in trying to estimate the true coherence. 
We define the sample auto- and cross-correlation coefficients by 


fij(v) = cn. 


We next state a theorem containing the basic statistical properties of the 
sample mean and autocovariances. We delay consideration of the sample spec- 
tral density until the next section when we discuss smoothing it to obtain 
consistent spectral estimators. As in the univariate case, the results for the 
sample autocovariance function assume that the process mean # either is O or 
is known and has been subtracted. The results for unknown mean and sub- 
tracting X are essentially the same. Proofs of these results can be found in 
Hannan (1970) or Brillinger (1975), for example. 


Theorem 4.2.1 | PROPERTIES oF X AND R 


Let X(1),...,X(n) be a sample realization from a covariance stationary 
bivariate time series having autocovariance function R. Then 


a) If for i,j = 1,2, we have Jim Rjj(v) = 0 then 


lim E(X —p)"(X — p) = 0, 


n->co 
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while if R. is absolutely summable, we have 


lim nE(X — p)? (KX — p) = fir (0) + fo2(0). 


n-+Cco 


If X is a general linear process with independent errors and coefficient matrices 
that are absolutely summable, then 


aN “£(0)). 


b) If X is a general linear process with independent errors and coefficient 
matrices that are absolutely summable, then Ri; and p;;(v) are consistent es- 
timators of R,;(v) and p;j(v). If X; and X>2 are univariate general linear pro- 
cesses with independent errors and absolutely summable coefficients, and the 
two univariate error series are independent, then the vector (/12(j), p12(k))* is 
asymptotically normal with mean zero and 


foe] 


Jim, nCov (f12(l), A12(m)) = D7 prr(v)p22(v +m — 1). 


v=>-—-0o 


If X is a bivariate Gaussian process with absolutely summable autocovariance 
function, then the bivariate analog of Bartlett’s formula is given by 


oo 


lim nCov (A12(!), Ar2(m)) = D> [pra(v)p22(v + m1) + pr2(v + m)pri(v - 1) 


m—>co 
uvu=-—0o 


— pi2(!) {p11 (v) pai(v + m) + p22(v)p21(v — m)} 
— pi2(v){p11(v) p2i(v +!) + p22(v)p21(v — m)} 


+ pr2(l)pr2(om){5p2s(0) + peal») + spa(u)}]- 


If X, and Xy are not cross-correlated, this reduces to 


Cov (A12(!), Pi2(m 7% pii(l)p22(m 
which gives 
i -le 
Var (f12(m)) = % 28 pii(v)p22(v). 


Implications: Part (a) allows us to find confidence regions for the vector p. 
Since 


n/2(X — p) + No(02, f(0)), 
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we can write rs 2 ¥s 
n(X — p)7#-1(0)(K — p) % 42, 
and the set of two-dimensional vectors h satisfying 


n(X —h)?£-1(0)(X—h) < x3. 


is a 100(1 — a)% confidence region for . The fact that (0) is unknown can 
be overcome by using a consistent estimate of it. For large samples, this has 
no effect on the confidence level. 

Part (b) of the theorem contains the crucial fact that the variance of the 
sample cross-correlation coefficient is a function of the autocorrelation functions 
of the two series. For example, if X; and X> are independent AR(1) processes 
with coefficients a and respectively, then we have 


pi1(v) =(—a)” and p22(v) = (—8)”, 


which means that 


Var(A12(v)) = oe 


This gives an approximate 95% confidence interval for Pi2(v) as 


2 


Pi2(v) + sates), 


For n = 100, a = —.9, and @ = —.8, this interval is Pi2(v) + 0.38, which means 
that a sample cross-correlation as large as 0.38 would not be surprising. See 
Problem C4.3 for another example. 


Testing for Independence 


We have seen that two univariate series that are jointly covariance station- 
ary are uncorrelated if their cross-correlation function is zero for all lags. If the 
bivariate series is Gaussian, then this also means that the two series are inde- 
pendent. Thus a natural method for testing for independence of two univariate 
series is to test the hypothesis that their cross-correlations are zero. Unfortu- 
nately, we have seen that the sample cross-correlations can appear to be quite 
large when the true cross-correlations are zero if there is high autocorrelation 
in the univariate series. Traditionally, the solution to this problem has been 
to use the cross-correlations for a filtered version of the original series rather 
than the series themselves. This process of filtering the data prior to analyz- 
ing them is called “prewhitening” the data. The justification for prewhitening 
when testing for independence is given in the next theorem. 
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Theorem 4.2.2 | TESTING FOR INDEPENDENCE 


The jointly covariance stationary univariate time series X; and X2 are 
uncorrelated (and thus independent if they are Gaussian) if and only if the 
series Y;(t) = gi(L)X,(t) and Y2(t) = g2(L)X2(t) are uncorrelated, where g; 
and g2 are any two invertible filters. 


Proof: We can write Y(t) = G(L)X(t), where 


He Ee 0 
1 0 92(L) 


and thus by the bivariate Filter Theorem we have 
fy (w) = G(e?*”) fx (w)G* (e™). 


Now X, and X2 are uncorrelated if and only if fx (w) is diagonal for all w since 
fx and Ry are Fourier transforms of each other. But since G is diagonal, we 
have that fx is diagonal if and only if fy is. 


Thus a simple test for independence of two jointly Gaussian covariance 
stationary time series is to (1) fit autoregressive processes to the univariate 
series, (2) calculate the cross-correlations of the errors, and (3) test whether 
these cross-correlations are significantly different from zero. Under the null 
hypothesis, the cross-correlations are asymptotically independent and identi- 
cally distributed with mean zero and asymptotic variance 1/n. Thus if we test 
each of 2M + 1 cross-correlations (usually for lags —M to M), we should use a 
significance level of a!/?™+1 for each lag so that the overall significance level 
will be a. 

In Example 4.3 we give a macro for carrying out this procedure. In Figure 
4.4 we display the result of applying this macro to the gas furnace data set 
introduced in Chapter 1. The data set itself is also given in Figure 4.4. Note 
that we multiplied X, by three and then added 30 so that the features of the 
data can be seen. The values of m, alpha, and maxp that were used were 20, 
.05, and 10 respectively. The autoregressive models for the univariate series 
have orders 6 and 4 and coefficients 


a, = (—1.93, 1.20, —0.19, 0.13, —0.27, 0.11)” 
a2 = (—1.85, 0.84, 0.31, —0.26)”. 


All of the cross-correlations fall within the bands except those for lags 3-6. 
Thus we can conclude that the two series are not independent. If we look at 
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Cross-Coprelations and Confidence Bands 


a 18.0 “6.8 6. 18.00 30.0 


Figure 4.4. The Gas Furnace Data and the Cross-Correlations and 95% 
Confidence Bands for the Prewhitened Data. 


the data themselves, we can see that the input series is leading the output 
series by approximately 3-6 time periods. 


4.2.2. Nonparametric Spectral Density Estimation 


Given data from a bivariate time series, we can apply the methods of 
Section 3.2 to the sample autospectral densities iy and ee to obtain consistent 
estimates of f;; and f22. In this section we consider applying windows to the 
sample cross-spectral density ey Traditionally, the same smoothing is applied 
to all three functions fae fo2, and fe so that the sampling properties of the 
resulting smoothed estimators can be easily stated. 


The CROSSP Command 
The command 
CROSSP (rho12,rho21,R10,R20,rho120,M,Q,kernel ,fi2r,f12i) 


will calculate the real and imaginary parts of the smoothed cross-spectra at 
the g = [Q/2] + 1 natural frequencies using the truncation point entered in M 
and the kernel determined by the input integer kernel, which can be any one 
of the first five kernels as described in the WINDOW command. 

In Example 4.4 we give a macro that will calculate and display the coher- 
ence and phase spectra (together with the confidence intervals that we derive 
below) for a bivariate data set. The results of applying this to the prewhitened 
gas furnace data for the Parzen kernel having truncation point 36 are displayed 
in Figure 4.5. We will discuss the implications of these graphs later, but for 
now, note that the coherence is highest for low frequencies. Also note that the 
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1.50 Coherence and Confidence Linits 


1,38 
1.10 
0.98 
0.78 
0.58 
0.38 
0.18 
- 10 
~.30 


-.5 
le 0.10 0.20 0.38 0.4 0.50 


Phase and Confidence Linits 


0.0 0.10 0.20 0.30 0.4 0.50 


Figure 4.5. Coherence and Phase Spectra and Confidence Intervals for the 
Prewhitened Gas Furnace Data Using the Parzen Window with Truncation 
Point 36. 


phase spectrum is smooth in pieces, with a sudden jump from the top of the 
graph to the bottom. This is because the arctangent is taken modulo 27, and 
thus when the curve reaches the top of the graph, it will “wrap around” to the 
bottom. 


Sampling Properties of Sample Cross-Spectral Density 


In Section 3.1 we saw that the sample autospectral densities are asymptot- 
ically uncorrelated at different frequencies and are asymptotically proportional 
to x? random variables. In the extension to the bivariate case, we have two 
further questions: (1) How are f;; and f2 related, and (2) what is the large 
sample behavior of the complex valued random variable fio(w)? Notice that in 
the matrix f(w) there are a total of four distinct, real, scalar random variables, 
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namely ful), foo(w), €12(w), and G2(w), since 
fi2(w) = €12(w) = igi2(w) and for(w) = €12(w) + 1912 (w). 


We have the following results about these four random variables. 


Theorem 4.2.3 | PROPERTIES OF BIVARIATE PERIODOGRAM 


Let f be the bivariate sample spectral density function for a realization of 
length n from a general linear process having independent errors and coefficients 
satisfying 


> [By (k RI? <00, = i,f = 1,2. 


k=-—0o0 


If 0 < w; < we < 0.5 are either fixed frequencies or two natural frequencies, 
then the elements of f (w;) are asymptotically uncorrelated with the elements 
of f(w,), and if we let 


Z(w) = (fiw), foo(w), é12(w), Gr2(w))? 


for 0 <w < 0.5, then (omitting the argument w from each element) 


Jim Var(Z(w)) 
fi \fi2l? fiici2 fig 
fio fo2e12 fo2912 
t (fir for + c?, — gio] C12912 
Symmetric sl fisfe2 — chy + 974] 
=V. 


We will not consider the asymptotic distribution of the cross- spectral den- 
sity as we will use smoothed estimates to make inferences about the coherence, 
gain, and phase spectra. For this we have the following theorem. 
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Theorem 4.2.4 | BIVARIATE WINDOW ESTIMATORS 


Let ue 
i= ye A(z7)R(v)e?r 


be a smoothed periodogram estimator of f(w) based on a realization of length 
n from a general linear process having independent errors and coefficients that 
are absolutely summable. If M — co and M/n > ow asn + oo, and0 <u; < 
w, < 0.5, then the elements of f (w;) are asymptotically uncorrelated with the 
elements of f (wx). If we define 


Z(w) = (fi1(w), fo2(w), G12(w), Gr2(w))? 
Z(w) = (fir(w), fa2(w), c12(w), g12(w))”, 
then 
Fle) — 2(w)) £4 Na(04,79), 
where the asymptotic covariance matrix V is given in Theorem 4.2.3 and 


aie ie d?(u) du. 


—co 


This theorem allows us to make inferences about the autospectra and the 
co- and quadrature spectra. It also allows us to find by the continuous function 
theorem (see Theorem A.4.5) the asymptotic distribution of the quantities 
(such as coherence and phase) that are functions of the elements of Z. These 
are given in the next theorem. 


Theorem 4.2.5 | PROPERTIES OF SPECTRAL QUANTITIES 


Under the conditions of Theorem 4.2.4, we have: 


a) If the squared coherence is strictly positive at frequency w, then the 
smoothed estimate of the amplitude spectrum satisfies 


Vn/M (Aia(w) — Ara) + N(0,0%(w)), 


where : 
2 (w) = YAj2(w) 


3 (1+——). 
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b) If the squared coherence is strictly positive at frequency w, then the 
smoothed estimate of the phase spectrum satisfies 


Vn/M (412(w) - 412(v)) ++ V0, 03(w)), 


where 


A?,(w) 1 
o2(w) = lest Dad —1). 
He WE 
If W?,(w) = 0, then $12(w) is asymptotically uniformly distributed over the 
interval (—7, 7). 


c) If the squared coherence is strictly positive at frequency w, then the 
smoothed estimate of the coherence spectrum satisfies 


where 


Vn/M (Wa2(w) — Waal) => (0, 0%/(w)), 


oy (w) = 5 (1- Wia(w)). 


The small sample bias of the coherence estimate can be appreciable if X, and 
X2 have a large cross-correlation for nonzero lags. 


d) If Wi2(w) = 0, then 


where c = (n/M7y) — 1. 


Implications: These results provide large sample confidence intervals for the 
amplitude, phase, and coherence spectra given by 


where the ¢’s are evaluated by substituting estimates of unknown quantities 


into the formulas for the o?’s. Part (d) provides an easily applied test for 
zero coherence at individual frequencies. To illustrate this, we have Figure 
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Festatistis and Critical Value 
or Squared Coherence 

70.67 
63.68 
56.53 
49,47 
“.@ 
35.33 


A 9.10 0.28 0.30 0.4 0.50 


Figure 4.6. Transformed Squared Coherence and Critical Value for the F 
Test for the Prewhitened Gas Furnace Data. 


4.6, which consists of a graph of the transformed squared coherence for the 
prewhitened gas furnace data. Superimposed on the graph is the critical value 
of the F distribution. See Example 4.5 for the macro that generated this figure. 


4.2.3. Autoregressive Spectral Estimation and Forecasting 


As in the univariate case, autoregressive modeling is easily applied and as 
long as the data being analyzed come from a process which can be expressed 
as an infinite order AR process, one can use the the chosen model for spectral 
estimation or forecasting. 


The CORRAR2 Command 


If the order of the process to be fit is known, then the known-order form of 
the CORRAR2 command described in Section 4.1.5 can be used to find parameter 
estimates. If the order is unknown, then the command 


A=CORRAR2(RXO,RYO,rhoxy0,rhox,rhoy,rhoxy,rhoyx,M,iopt, 
n,p,SIGMA,cat,ier) 


can be used to estimate the order p and the parameters of the process. This 
form is the same as the known-order form except that M is the maximum order 
to fit, n is the sample size, cat is an output array of length M containing the 
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values of the bivariate CAT criterion (see Parzen (1977)) 
9 
T(k) = =o Dasa he 
CAT(k) = tr nde mae le 


where &; is the “unbiased residual variance” of order j 


~ n B 


n— 23 
and x; is the order-j “biased residual variance” estimator 
j 


= 5° A;()R(I) 


=0 


~ 


The input integer iopt is 1 or 2 depending on whether order M is to be used or 
the CAT criterion determined order less than or equal to M is to be used. 

Once the order and parameters of the AR process have been determined, 
then the ARSP2 command can be used to find the corresponding autoregressive 
spectral estimator. In addition, forecasts of future values of the process can be 
found by 


P 
X(n+h) = - So A( X(n+h-— J), 


pail 


where X(n +h—j) =X(n+h — j) if j > h. In Example 4.6 we give a macro 
that (1) calculates and displays estimates of the auto- and cross-correlations 
from a data set, (2) fits a bivariate AR model to the data, and (3) uses this 
model to find autespecteal estimates as well as estimates of the coherence and 
phase spectra. We applied this macro to both the original gas furnace data 


and the prewhitened gas furnace data and found the following correlations and 
AR models: 


Original Gas Furnace Data 


v rhoi(v) rho2(v) rhoi2(v) rho21(-v) 


te) 1.000 1.000 - .484 - .484 
1 -952 971 -.598 - .393 
2 -834 896 =s120 = .329 
3 -682 793 - .843 -.286 
4 -531 680 -.925 e260) 
5 -408 574 -.950 - .243 
6 -318 485 -.915 ae ea 
7 . 260 416 -.829 - .206 
8 +228 366 cee KA =.179 
9 -213 330 - .600 -.149 
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10. - 208 .307 -.495 ~.118 
Autoregressive Coefficient Matrices 
-1.925889 -001240 
-.050508 -1.299794 
1.201669 - .004222 
-020493 - 327703 
-.116929 - 008672 
-711791 . 257010 
-.104232 - .003270 
-.195398 ~ .133420 
Autoregressive Error Variance Matrix 
-035155 - .007313 
- .007314 .097240 


Prewhitened Data 


v rhoi(v) rho2(v) rhoi2(v) rho21(-v) 


0 1.000 1.000 -.041 -.041 
1 -010 269 -052 - .060 
2 -013 083 -000 -019 
3 -.010 -.001 ~ .288 007 
4 - .042 -.090 - .368 064 
5 -.004 =, 061 ~ .434 031 
6 -O77 -094 =.227 - .085 
tf -021 057 - .037 -.044 
8 -.074 O11 -095 ~ .087 
9 - .090 - .068 -113 -013 
10 035 007 - .026 -025 
Autoregressive Coefficient Matrices 
- .008204 -030892 
-.072571 . 303707 
- .024308 ~ .036640 
-019692 - 303947 
-005909 -000587 
-555159 - 169584 
.051469 - .027097 
- 853693 . 153809 
-013523 - .006732 
1.135349 -121737 
-.077010 -048511 
-921561 - .050637 
~ .051536 - .010818 
-647568 - .038759 
.049215 -041825 
- 311347 ~ .042690 
Autoregressive Error Variance Matrix 
-033566 - .001345 
- .001345 -054038 


The prewhitening has indeed removed the autocorrelation in the two data 
sets, while the cross-correlation remains high for lags 3-6. An important thing 
to notice is that the prewhitening has greatly decreased the cross-correlation 
for negative lags, and the cross-correlations for the prewhitened data clearly 
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1 ao Squared Coherence Spectrum Preuhitened AR Squared Coherence 
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0.88 0.8 
0.68 0.68 
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Figure 4.7. Phase and Squared Coherence Estimates for the Original and 
Prewhitened Gas Furnace Data. 


reveal that the dependence of the output to the input is as a delay process. In 
Figure 4.7, we give the squared coherence and phase estimates for the two data 
sets. The linear shape of the phase confirms the delay nature of the dependence 
(see Section 4.1.4). Note that one should also estimate the coherence after the 
two series have been aligned (see Problem C4.4), because as stated in part (c) 
of Theorem 4.2.5, the presence of large cross-correlations at nonzero lags can 
cause a bias in the coherence estimator. In any event, the coherence is clearly 
significantly different from zero, and in the next section we discuss modeling 
the dependence of the output series on the input series. 


4.2.4. Estimating the Transfer Function of a Filter 


In Section 4.1.4 we considered approximating the optimal filter for one 
univariate series based on another in the case where the true auto- and cross- 
spectra were known for the series. In this section we consider substituting 
estimates of the spectra into the procedures for the known spectra case. For 
another approach to the problem of estimating filter coefficients, see Chapter 
11 of Box and Jenkins (1970). 

Given data X(1),...,X(n) from a covariance stationary bivariate time 
series X, we consider the following procedure for expressing X2(t) as a filtered 
version of X: 


1. Prewhiten X, and X> using univariate, CAT criterion determined order 
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AR filters. Call the two orders p; and pz and the corresponding polyno- 
mials g, and gp. Thus the result of this stage is the two error series 


€1(t) = g:(L) Xi (t) and €2(t) = G2(l) Xo(t). 


If the two orders are different, then we align the two error series so that 
the result is a bivariate series e of length n — max(pi, po). The macro 
PREWHITE (see Example 4.3) performs this procedure. 


. In this step we use either nonparametric or autoregressive spectral esti- 


mation to find estimates of the auto- and cross-spectra of the e series. 
We then substitute these estimators into the procedure (see the FIND- 
FILT macro in Section 4.1.4) for approximating the integrals that find the 
coefficients of the filter for e2 as a function of e}. 


. The effect of the first two steps of the procedure is to write 


§2(L)X2(t) = h(L)§.(L) X(t), 
where A(L) is the polynomial of the filter found in step 2. Thus we have 
Xo(t) = G3 *(L)A(L)51(L) Xi). 


The final step of the process is to multiply the three polynomials together 
to obtain the final filter expressing X2 as a function of Xj. 


The following macro will carry out this process. To use it, the user must 


enter the input and output series and their length in the variables x, y, and 
n, the maximum AR order to be allowed in the prewhitening procedure in 
maxp, the number of frequencies and truncation point for the nonparametric 
estimator in Q and nt, and finally the number of filter coefficients to find in m. 


GCNAaA HK WON 


© 


as FILTEST.MAC: This macro estimates the filter coefficients 

ay relating an output series y to an input series x. 

i3 It is assumed that only coefficients with nonnegative 
He lags are in the model. 


as The series are first prewhitened (via the PREWHITE 
rie macro). Then the auto- and cross-spectra are found 
ae for the prewhitened series. Then the FINDFILT macro 
eas is used to estimate the filter coefficients. 

te Then the filter for the original data is found. 


es INPUT: x,y,n (original series and their length) 
te maxp (maximum AR lag for prewhitening) 
of Q,nt (number of frequencies and truncation point in spectra) 
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16 m (number of filter coefficients to find) 
nT es 

18 PAUSE 

19 ;start 

20) as 

21 PROMPTOFF 

22 MACRO(prewhite,start) 

23 ; 

24 CORR2 (xx, yy,nn,nt ,1,rx0,ryO,rhoxy0,rhox, rhoy ,rhoxy ,rhoyx) 
25 fxx=WINDOW(rhox,rx0,nt,Q,4) 

26 fyy=WINDOW(rhoy,ryO,nt,Q,4) 

27 CROSSP(rhoxy ,rhoyx,rx0,ry0,rhoxy0,nt,Q,4,fxyr,fxyi) 
Zee 

29 MACRO(findfilt ,start) 

30 ; 

31 beta=beta/beta0d 

32 gama=MULTPOLY (beta, alphax ,m, px) 

33 mppx=mtpx 

34 betay=INVPOLY(alphay ,py ,40) 

35 gama=MULTPOLY (betay , gama, 40,mppx) 

36 gama=gama*beta0 

37 LABEL(gama)=’Final Filter Coefficients’ 

38 LIST(gama,m) 

39 list (beta0d) 

40 PROMPTON 


To illustrate the use of this macro, we used the gas furnace data with 
maxp=10, Q=128, nt=30, and m=10, and found the following filter coeffcients: 


Coefficients From FILTEST for Gas Furnace Data (beta0=-0.069691) 
1 | .084685 -.036375 -.495704 -.591778 -.712478 
6 | -.411839 -.285202 -.099515 -.042972 -.061627 


These agree rather well with the model found by Box and Jenkins (1970), 
p. 399 using parametric methods: 


(1 — 0.57L) X(t) = —(0.53 + 0.37L + 0.51L?) X(t — 3), 


since if we invert 1 — 0.57L and multiply it times the polynomial on the right 
hand side of the model, we get 


Coefficients of Box-Jenkins Model For Gas Furnace Data (beta0=0) 
1 | .000000 -000000 -.530000 -.672100 -.893097 
6 | -.509065 -.290167 -.165395 -.094275 -.053737 


See Problem C4.5 for another illustration of the use of the FILTEST macro. 
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4.3. Other Topics in Bivariate and Multiple Time Series 


In the first three chapters of this book we presented a wide variety of 
models and estimation procedures for univariate time series. In this chapter 
we have emphasized nonparametric and autoregressive spectral estimation and 
autoregressive prediction. These methods are well established and are widely 
used in science. It is hoped that as personal computers become more powerful 
(particularly in the area of having access to more random access memory), we 
can revise TIMESLAB to incorporate more of the analogs from the univariate 
case, as well as extend the bivariate commands to become multivariate com- 
mands. Most of the ideas of bivariate time series analysis extend directly to 
the multiple case, where instead of observing only two variables at each time 
one observes d > 2 variables. Thus the autocovariance and spectral density 
functions become (d x d) matrices, as do AR coefficients and filter coefficients 
in general. The books by Priestley (1981) and Brillinger (1975) provide a good 
description of the general area of multiple time series. 


4.4. Examples and Problems 


Example 4.1 FORMING TABLE 4.1 AND FIGURES 4.1 AND 4.2 


The data and correlations that are discussed in Section 4.1.1 come from a 
bivariate autoregressive model (see Section 4.1.5). The ARCORR2 command can 
be used to find the correlations for such a process while the ARDT2 command 
can be used to simulate a realization. The macro FIG41.MAC was used to form 
Table 4.1 and Figure 4.1: 


esis 

2 3% FIG41.MAC: macro to form Figure 4.1 
3 55 

4 Ost; INPUT: none 

Sass 

6 PAUSE 

7 ;start 

8 


a=<-.6,-.4,.5,-.5> 

sigma=<1,0,0,1> 

10 M=12 

11 ARCORR2(a,1,sigma,M,ri0,r20,rho120,rhoi1,rho2,rho12,rho21 ,ier) 
12 MACRO(corr2plt,start) ;produce table and plots 


vo} 


i 

4a Generate and Plot Realizations: 
Lo: 3 

16 ;data 


17 x1=ARDT2(a,1,sigma,0,60,ier) 
18 x=EXTRACT(x1,1,120,2,n) ;every other one starting with ist is X1 
19 y=EXTRACT(x1,2,120,2,n) ;every other one starting with 2nd is X2 
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20 
21 
22 
23 
24 
25 
26 
27 
28 
29 


n=60 

y=y-4 

x=x+4 

yy=<y ,x> 

ii=LIWE(60,0,1) 

xx=<ii,ii> 

type=<12 ,12> 

LABEL(yy)=’Realization From Bivariate Process’ 
LABEL(xx)=’X1 is on the Top’ 

PLOTK (xx ,yy ,60,2,type,0,60,-10,10) 


Note that FIG41 uses the macro CORR2PLT.MAC to actually produce a table 


of correlations (which was saved in a file using the RECORD command, edited 
slightly, and then inserted into the file that produced this book) and the graphs 
of correlations and data sets: 


COCMONA AR WH PR 


NHN NY NNN NNN RP RP RP BPP BP ee BP 


a) 

8 CORR2PLT.MAC: macro to display auto and cross-correlations 
; , 

3; INPUT: rho120 (cross-correlation of lag 0) 

as M (number of lags at which correlations have been found) 
AN rho1, rho2 (arrays containing autocorrelations) 

rt rhoi2, rho21 (arrays containing cross-correlations) 
5 

PAUSE 

;start 

; 

: Construct the Table of Correlations: 

mp1i=M+1 

ind=LIWE (mp1 ,-1,1) 


rr=<ind,1,rhoi,1,rho2,rho120, rho12,rho120,rho21> 
rr=TRANS (rr ,mpi,5) 

LABEL(rr)=’Table of Auto and Cross Correlations’ 
mp15=mp1i+*5 

form=’f3.0,4f9.3’ 

LIST(rr,mp15,5,form) 

PAUSE 

- Autocorrelation Plots: 

PLOT(rho1,M,0,M,-1,1) 

PLOT(rho2,M,0,M,-1,1) 

: Construct the Cross-Correlation Plot: 
ind=LIWE(M,0,1) 

ii=-ind 

ii=REVERSE (ii ,M) 

ind=<ii,0,ind> 
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36 rr=rho21 

37 rr=REVERSE(rr ,M) 

38 rr=<rr,rhoi20,rho12> 

39 rr=<rr,-1,1,0,0,1,1,1,-1> 

40 MM=-M 

41 m2p1=2¢M 

42 m2p1=m2pit1 

43 ind=<ind,0,0,MM,M,MM,M,M,M> 
44 nn=<m2pi ,2,2,2,2> 

45 type=<13,2,2,2,2> 

46 LABEL(rr)=’Cross-Correlations’ 
47 LABEL(ind)=’ ” 

48 PLOTK(ind,rr,nn,5,type,MM,M,-1,1) 


Notice how the PLOTK command is used in FIG41 and CORR2PLT. Of par- 
ticular interest is how the vertical and horizontal lines are superimposed on 
the cross-correlation plot, and how the individual points on the data plot can 
be seen by using type=<12,12> (see the PLOTK command). Note also that the 
ARDT2 command generates a bivariate realization of length n in an array of 
length 2n, in which the elements of the first univariate series are the odd num- 
bered elements and the second univariate series are the even numbered ones. 
The EXTRACT command is used to form individual arrays for the two univariate 
series. To obtain Figure 4.2, we used the ARSP2 command to find auto- and 
cross-spectra and then used the macro CROSSP.MAC given in Section 4.1.2 to 
calculate and plot the coherence and phase spectra. 


Example 4.2 INTERPRETING COHERENCE 


The macro COHER.MAC was used to perform the experiment in Section 4.1.2 
that illustrated the interpretation of the coherence function. Notice how the 
frequency for a scatterplot is inserted into its plotting label. 


Shin 

2 ;;  COHER.MAC : This macro generates nsamps samples of length n 
bee He for a bivariate AR process and for each one finds 
ed the univariate periodograms at the n frequencies 
5 35 between O and 1. A scatterplot of the amplitudes 
Giaeg for frequency (nf-i)/n is then produced. 

T it 

Pa INPUT: n,nsamps ,nf ,seed 

9 35 

10 ;; 

11 PAUSE 

12 ;start 

13 PROMPTOFF 

14 a=<-.6,-.4,.5,-.5> 

15 sigma=<1,0,0,1> 

16 c1=LINE(nsamps) ;reserve space for amplitudes 
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17 c2=LIWE(nsamps) 


18 x=WH(seed,10) ;warm up random number generator 

19 ns=1 ;initialize sample counter 

20 ; 

al Set 

ae 

23 LIST(ns) 

24 x=ARDT2(a,1,sigma,0,n, ier) ;generate bivariate AR sample 
25 n2=2+n 

26 x1=EXTRACT(x,1,n2,2,nx) ;pull off the univariate series 
27 x2=EXTRACT(x,2,n2,2,nx) 

28 rho=CORR(x1,n,0,n,1,r01,f1) ;find periodograms 

29 rho=CORR(x2,n,0,n,1,r02,f2) 

30 cifns]=filnf] ;pull off amplitude for desired frequency 


31 c2[ns]=f2[nf] 

32 IF(ms.eq.nsamps, end) 

33 ns=nsti 

34 GOTO(s1) 

355 

36 ;end 

Dias 

38 mnsamps=-nsamps 

39 LABEL(c2)=’Scatterplot of Amplitudes’ 
40 LABEL(c1)=? »° 

41 PLOT(ci,c2,mnsamps) ;do scatterplot 


Example 4.3 TESTING FOR INDEPENDENCE 


The macro INDTEST.MAC can be used to test for independence of two uni- 
variate time series as described in and below Theorem 4.2.2. 


IK INDTEST.MAC : Macro to test two univariate series for 

ry independence. AR processes are fit to each 
i series and the cross-correlations of lags 
-m to m graphed with 100(1-alpha)% 

Sin confidence bands. 


-F INPUT: x, y, n (the two series and their length), m,alpha 
0h maxp (maximum AR order allowed) 


COMNAA Hh WH eS 


= 
oO 


PAUSE 

;start 

13 MACRO(prewhite,start) 

14 CORR2(xx,yy,nn,m,1,rx0,ryO,rhoxy0,rhox,rhoy,rhoxy ,rhoyx) 

15 mm=-m 

16 rhoyx=REVERSE(rhoyx,m) 

17 rho=<rhoyx,rhoxy0,rhoxy> ;rho has the correlations for lags -m to m 
18 4 

iD ; Find Z-value to use: 


He 
Ne 
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mi=2*m+1 

alphi={1+{1-alpha}~{1./m1}}/2 

z=dist(z,3,1,alph1) 

, 

; Construct input to PLOTK: 

Hy 

on=nn~.5 

up=z/on ;up and low are limits for confidence bands 
low=-up 

mmm1=mm-1 

x1=LIWE (m1 ,mmmi ,1) 

x1=<x1,mm,m,mm,m> 

yi=<rho,up,up,low,low> 

type=<33 ,2,2> 

leng=<m1 ,2 ,2> 

LABEL(x1)=? ? 

LABEL(y1)=’Cross-Correlations and Confidence Bands’ 
PLOTK (x1 ,y1,leng,3,type,mm,m,-1,1) 


Notice that INDTEST uses the macro PREWHITE.MAC to do the prewhitening 


Process. 


OoOmMNA AE WN 


WON NNN NN NN NWN FFP RPP RP RP RP RP ee 
cweomnNtoanhwnroowmonaoanrh Wnr Oo 


at PREWHITE.MAC : Macro to use AR filters to prewhiten and align 
33 two univariate series. 


ais INPUT: x, y, n (the two series and their length), 
ay maxp (maximum AR order allowed) 


ze OUTPUT: xx,yy,nn (the resulting series and their length) 
>? 

PAUSE 

;start 

xx=SUBMNS(x,n,1,xbar) ;subtract means 
yy=SUBMUS(y,n,1,ybar) 
alphax=DTAR(xx,n,maxp,2,2,1,px,rx0,rvarx) ;fit AR processes 
alphay=DTAR(yy ,n,maxp,2,2,1,py,ry0,rvary) 
LIST(px,py,rx0,ryO,rvarx,rvary) ;px=x order, py=y order 


IF (px.gt.0) ;find residuals 
xx=FILT (xx, alphax,1,n,px) 

ENDIF 

IF (py. gt .0) 
yy=FILT(yy , alphay ,1,n,py) 

ENDIF 

nx=n-px ;align the residuals 

ny=n-py 

nn=nx ;nn will become the common length of residuals 

IF(nx.eq.ny,end) ;if same length, we’re finished 

IF (nx.gt.ny) ;handle case where x residuals longer 
nn=ny ;(delete first part of x residuals) 
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31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 


ni=nx-ny 
ni=ni+ti 
xx=EXTRACT (xx ,n1,nx) 
ENDIF 
IF (nx.1t .ny) shandle case where y residuals longer 
nn=nx ;(delete first part of y residuals) 
ni=ny-nx 
ni=niti 
yy=EXTRACT (yy ,n1 ,ny) 
ENDIF 
j;end 
; 


Example 4.4 | NONPARAMETRIC SPECTRAL ESTIMATION 


The macro CROSSPE.MAC can be used to find nonparametric estimates of 
the phase spectrum and squared coherency spectrum and cofidence intervals 


for each given a data set. 


C©CmWMWN ADAH WH 


a 0 Sl el 
SO OANAAhWNHRP OH WNAA HR WNH OO 


ae CROSSPE.MAC: This macro will compute the smoothed coherence and 
>> phase spectrum for two series and put confidence 
33 intervals on them. These are not confidence bands. 


ais INPUT: x,y,n (two series and their length) 
m, kernel (scale parameter and kernel option) 
a8 Q (number of frequencies between O and 1) 
15 

PAUSE 

;start 

gamm=<2 , .66667,.795,.539,.586> ;see Table 3.3 
CORR2(x,y,n,m,1,RXO,RYO,rhoxyO, rhox ,rhoy ,rhoxy ,rhoyx) 
fxx=WINDOW(rhox,RXO,m,Q,kernel) 
fyy=WINDOW (rhoy ,RYO,m,Q,kernel) 

CROSSP(rhoxy ,rhoyx,RXO,RYO,rhoxy0,m,Q,kernel ,fxyr,fxyi) 
q={Q/2}+1 


freq=LIWE(q,0,.5,1) ;get frequencies for plots 
POLAR(fxyr,fxyi,q,amp,phase) ;now have phase spectra 


coher=amp/{fxx*fyy}~ .5 ; coherence 

FE Do coherence: 

se={{m+gamm[kerne1] /{2.+*n}}*{1-coher72}}7~.5 
cohlow=coher-2#se 

cohup=coher+2*se 

ff=<freq ,freq,freq> 
cc=<coher,cohlow,cohup> 
label(cc)=’Coherence and Confidence Limits’ 
LABEL (ff )=’ : 
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31 lengs=<q,q,q> 

32 type=<2,2,2> 

33 PLOTK(ff,cc,lengs,3,type,0,.5,-.5,1.5) 
a4; 

35 ; Do phase: 

36 ; 

37 fac=amp~2*{1-coher~2}/{coher~ 2} 

38 se={{m*+gamm[kernel]/{2.*n}}*fac}~.5 

39 phlow=phase-2*se 

40 phup=phase+2*se 

41 cc=<phase,phlow,phup> 

42 LABEL(cc)=’Phase and Confidence Limits’ 
43 PLOTK(ff,cc,lengs,3,type,0,.5,-4,4) 


Example 4.5 TESTING FOR ZERO COHERENCE 


In Example 4.3 we gave a macro that can be used to test for independence 
of two univariate series based on the cross-correlation function. —The macro 
COHTEST.MAC provides a test for independence based on the squared coherence 


function. This test is based on part (d) of Theorem 4.2.5. 


2S COHTEST.MAC: This macro will test the hypothesis of 0 
34 coherence at each of the [Q/2]+1 
$3 frequencies between O and .5. 


33 INPUT: x,y,n (two series and their length) 
ys m, kernel (scale parameter and kernel option) 
Hi Q (number of frequencies between O and 1) 


omnNaanh WN eK 


PAUSE 

;start 

gamm=<2, .66667, .795, .539,.586> ;see Table 3.3 
CORR2(x,y,n,m,1,RX0,RYO,rhoxy0,rhox,rhoy ,rhoxy, rhoyx) 

£xx=WINDOW (rhox,RXO,m,Q,kernel) ;window auto- and cross-spectra 
fyy=WINDOW(rhoy ,RYO,m,Q,kernel) 
CROSSP(rhoxy,rhoyx,RXO,RYO,rhoxy0,m,Q,kernel ,fxyr ,fxyi) 


;plot F-test: 


Ce ee 
own mNoOanrh WN FH OO 


w2={fxyr”2+fxyi~2}/{fxx*fyy} ;squared coherence 
c={n-m*gamm[kernel]}/{m+gamm[kernel]} ; 
fstat={c+*w2}/{1-w2} 

nc :=2*c ;round off degrees of freedom 
ft=DIST(£,3,1, .95,2,nc) ;find critical value 
q={Q/2}+1 ;number of frequencies 
freq=LIWE(q,0,.5,1) ;get frequencies for plots 
ftab=LIVE(q,ft ,0) 

LABEL (fstat)=’F-Statistic and Critical Value’ 
LABEL(ftab)=’For Squared Coherence’ 

PLOT2(ftab,fstat ,q,q,1,1,freq,0,.5,0,5) 


WON NN NNN NN WY 
coe mn anh WN FE 
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Example 4.6 BIVARIATE AUTOREGRESSIVE MODELING 


The macro BAR.MAC carries out the bivariate autoregressive model deter- 


mination and cross-spectral estimation procedure described in Section 4.2.3. 


omntoauwnh WW eH 


NNNNNNNNNEP RPE PB ee eee 
OBIAAAPWNEFOHOWNAAAWNHRO 


aie BAR.MAC: macro to perform bivariate AR model fitting and 
+3 spectral estimation. 


3; INPUT: x, y, n (data and sample size) 

3 maxp (maximum AR order) 

3e Q (number of frequencies between 0 and 1) 

PAUSE 

;start 

CORR2(x,y,n,maxp,1,r10,r20,rho120,rho1 ,rho2,rho12,rho21) 

M=maxp 

MACRO(corr2plt,start) ;form table and plots of correlations 

. Fit AR: 

;ar2 

A=CORRAR2(r10 ,r20,rhoi120,rhoi ,rho2,rho12,rho21,maxp,2,n,p,sigma, cat, ier) 
LIST(p) 

IF (p.eq.0,end) 

LISTM(A,2,2,p) 

LISTM(sigma,2,2) 

> 

; AR Spectra: 

ARSP2(A,p,sigma,Q,f11,£22,f12r,f12i) 

MACRO(crossp,start) ;this plots autospectra, coherence, and phase 
;end 


Computational Problems 
9 a Se Se. Pe Oe OT ee 


C4.1. Use the FIG41 macro to generate a realization of length 60 from the 
process discussed in Section 4.1. Produce a scatterplot of X; (¢) and X2(t + 2) 
for t = 1,...,58 and a scatterplot for X\(t) and X9(t — 2) for t = 3; 28, 00k 
Are these plots consistent with the discussion of Figure 4.1? 


C4.2. Write a macro that will generate 100 realizations of length 100 from a 
bivariate process 


Xo(t) = X1(t) + .5Xi(t — 1), 
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where X; ~ WWN(1), and for each realization calculate the periodograms of 
X, and X»2 (use the CORR command for each one). For a user-specified integer 
nf, store all 100 pairs of nfth values of the periodogram, and after all 100 
pairs are generated, produce a scatterplot. Try out the macro for nf=10 and 
nf=40. Are the results consistent with the fact that the coherence spectrum 
is identically one for this process? 


C4.3. If X; and X2 are independent AR(2) processes with coefficients 0.4, 0.5, 
and 0.3, 0.9, respectively and error variances 1, approximate Var(/2(1)) for a 
realization of length 100. (Hint: See part (b) of Theorem 4.2.1.) 


C4.4. Align the prewhitened gas furnace data so that the resulting cross- 
correlation is largest for lag 0 (write a macro that will find the cross-correlations 
for a specified shift), and then use the CROSSPE macro to find the estimate of 
the squared coherence. Does the alignment make any difference relative to 
Figure 4.7? 


C4.5. The following macro will simulate a realization from a specified transfer 
function model: 


gl BA 

Bg SIMFILT.MAC: macro to simulate the transfer function model 
3.05); 

Ass X2(t) = bO*Xi(t) + Sum(j=1,K) b(j)Xi(t-j) + eps(t) 
5 33 

Gas where Xi is an AR(p) process with coefficients 
6 ie alpha, and eps is WH(sig2) 

8 55 

9 Gee INPUT: n (length of realization) 

Gi alpha, p, sig2 (variance of WH) 

LL K, bO, b (array of length K with coefficients) 

i et seed (random number generator seed) 

13 ;; 

14 ;; OUTPUT: x1, x2 (arrays of length n containing Xi and X2) 
ass 

16 PAUSE 

17 ;start 

18 n=ntK 


19 x1=ARDT(alpha,p,1,seed,n,ier,r0) 

20 x2=FILT(xi,b,b0,n,K) 

21 kpi=K+1 

22 x1=EXTRACT(x1,kp1i,n) ;pull off corresponding part of x1 

23 n=n-K 

24 e=WH(0,n) 

25 x2=x2+{sig2~.5}*e 

Use this macro to simulate a realization of length 100 when Xj is the 

AR(2) process with coefficients 0.3 and 0.9, and the filter coefficients are 


Bo =; (By = 0,75, 185 = 0.50," Bs = 0.25. 
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Use the FILTEST macro to estimate these coefficients. Use Q = 64 and Q = 128 
and see if it makes any difference. 


C4.6. Compare the square modulus of the frequency transfer functions of the 
two sets of filter coefficients that we discussed for the gas furnace data. 


Theoretical Problems 
Todt SN Seek Set ERAN ee) SN See ee 


T4.1. Suppose that X(t) = (Xi(t), X2(t))? is a bivariate time series having 
autocovariance function R and spectral density function f. Define the univari- 
ate series Y by 


Y(t} She. (t) + h2Xo(t). 
a) Show that the autocovariance function of Y is given by 


R(v) = hy Rui (v) + h2R22(v) + hyho2Rio(v) + hyh2Ro1(v). 


b) Thus show that the spectral density function of Y is given by 
fy (w) = h7f(w)h, 

where h = (hy, h2)?. 

T4.2. If X(t) = (Xi(t), X2(t))” is a bivariate time series having spectral 
density f and Y;(t) = g(L)Xi(t) and Yo(t) = h(L)Xo(t) are filtered ver- 
sions of X; and X92 respectively, then show that the coherency spectrum of 
Y(t) = (Yi(t), Yo(t))” is the same as that for X. (Hint: Use the bivariate 
Filter Theorem to find the auto- and cross-spectral densities for Y.) 

T4.3 Verify that the expression for fy is correct in part (b) of Theorem 4.1.4, 
and show that the expressions for Ry and fy reduce to those given in the 


univariate Filter Theorem if Y and X are univariate series. 


T4.4. Suppose that Z,; and Z2 are two uncross-correlated white noise series, 
each having variance 1, and we let 


Xo(t) = Z(t) + PoZi(t-1)+ Z(t), Xi (t) = Z,(t), 


which we can also write as 


X2(t) = A, X1(t) + B2Xi(t — 1) + Z2(t), 
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where X, and Zp» are uncross-correlated white noise series. Show that 
Ryo(v) = E[Xy(t) (A1Xi(t + v) + PoXi(t + v— 1) + Zo(t + v))] 
= Ai Rii(v) + B2Rir(v — 1) 
fry v= 
ARG Sn DO 


0, otherwise 


and 
fio(w) = Bit Boe?" = Bi + Bo cos 2mw — io sin 2rw 
and 
€12(w) = Bi + Bo cos 27w, gi2(w) = Bosin2rw, 

while 

Aia(w) = / Bf + BF + 281 Bo cos 2nw 
and By sin? 

= — Po sin 27rw 
¢12(w) = arctan (eS fg ee ~-) ‘ 


T4.5. What is the gain spectrum for the pure delay process discussed in 
Section 4.1.4? 


T4.6. Show that 


1 } 
fi2(w) e2tijw des i 
0 


C12(w) : 
cos 2 dw, 
o fii(w) 11(w) Misr f 


f 


where cj2 is the cospectral density of X; and X>. 


T4.7. If X1 is a white noise process with variance 1 and 


show that the cross-spectrum 


4 (1— emt) 


ie 1 ¢—2niw |?” 


fia(w) = 
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(Hint: Recall that if z is a complex number, then 1/z = z/|z|?.) 
T4.8. Express the coefficients of the fourth-degree polynomial 

g(z) = |In + A(1)z + A(2)z?| 


in terms of the elements of the coefficient matrices A(1) and A(2) for a bivariate 
autoregressive process of order 2. 


T4.9. Show that the bivariate sample spectral density can be written as 
n 


f(w) = ~ (x: (X(t) — X) ir (x: (X(s) — X) ei) 


3=1 


T4.10. Show that a univariate AR(p) process X can be expressed as a first- 
order autoregressive process Y of degree d; that is, if Y7(t) = (X(t), X (t- 
1),...,X(¢— p+1)) and e7 (¢) = (c(t), 0,...,0), then 

Y(t) + BY(t — 1) = €«(t), 


where the (p x p) matrix B (which is called the companion matrix) is given by 


a1 a2 3... @p-1 Op 
melee ae Ue eae 0 0 
B21 OMeIMO. ..42%0- 10 
De eae a ee] ae) 


T4.11. Suppose X is a periodic autoregressive process of period d, orders 
Pi;---»Pa, Coefficients a,(j), and residual variances o?,... ,07. Define the d- 
dimensional time series Y by letting Y(1) consist of X(1),... ,X(d), Y(2) be 
the next d X’s, and so on. Show that Y is a d-dimensional autoregressive 
process of order p, (d x d) coefficient matrices A(1),...,A(p), and residual 
variance matrix ©, where 

ee ah 


1 
d ts 


p= 


while © = L-1DL-7, A(j) = LB); Di= Diag(o?,...,07), Lis a (d x 
d) unit lower triangular matrix having a,(k — j) as its (k,j)th element, and 
Byj(v) = ax (dv — j +k), which is zero if du —j +k > pr. 


APPENDIX A 


Some Mathematical and Statistical 
Topics 


In this appendix we discuss several ideas from mathematics and statis- 
tics that are used in the book. We begin with a description of some matrix 
operations. 


A.1. Matrix Operations 


In this section we consider three operations on matrices: (1) the modified 
Cholesky decomposition of a positive definite matrix, (2) the Gram-Schmidt 
decomposition of a nonsingular (n x m) matrix, and (3) the matrix sweep 
operator. Each of these is useful as both a computational and theoretical tool. 


A.1.1. The Modified Cholesky Decomposition 


An (n x n) matrix V is said to be positive definite or positive semidefinite 
if for any nonzero n-dimensional vector h, we have h? Vh > 0 or h? Vh > 0 
respectively. In statistics it often necessary to find what is called the square 
root of a positive definite matrix. 


Definition. The square root of a symmetric, positive definite (n x n) matrix 
V is an (n x n) matrix A satisfying V = AA7. We denote such a matrix by 
V1/2_ The inverse square root of V is the inverse of V'/? and is denoted by 
V~-1/2_ The transposes of V~! and V~1/? are denoted by V~? and V-T/? 
respectively. 


The next theorem shows that there exists a unique matrix square root 
for any positive definite matrix and also shows how to find it. Note that a 
triangular matrix is called unit if it has ones on the main diagonal. 
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Theorem A.1.1 | MODIFIED CHOLESKY DECOMPOSITION 


Let V be a symmetric (n x n) matrix. Then 


a) V is positive definite if and only if there exists a unique unit lower 
triangular (n x n) matrix L and a unique diagonal (n x n) matrix D having 
positive diagonal elements, such that 


V=LDL’. 


b) This factorization is called the modified Cholesky decomposition (MCD) 
of V, and if the decomposition exists we can calculate the elements of L and 


D one row at a time by D, = Vi; and fori = 2,...,n: 
Wipe yrs > Pe Dye 
Lj = et ee Dizi bitDu LEP Al pete set 
D;; 
Feant 


Dj = Vii — 2 Dy Li. 
t=1 


c) The MCD of V is nested; that is, if Vx, Lx, and D, represent the upper 
left-hand (k x k) parts of V, L, and D, respectively, then 
Vr— LD, Ue. k=1,...5n, 


and thus for any k greater than or equal toi and j, the (i, j)th elements of Vx, 
L,, and D,, can be denoted by Vi;, Lij, and D;;, respectively. 


d) The unique square root of a positive definite matrix V is given by 


Vi/?2 — LD'/? where D!/2 = Diag(D}!?, far, Ray 


e) The inverse square root of a positive definite matrix V is given by 


V-¥? = D-/?L-1 where D~¥/? = Diag(D;,/, ..., Dad/?). 


The MCHOL Command 
If the array A contains the symmetric (px p) matrix A, then the command 
MCHOL(A,p,L,D, ier) 


will attempt to find the Cholesky factors L and D of A. If it judges that A 
is not positive definite, that is, if a diagonal element of D is found that is less 
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than 10~?°, it will return a 1 in the output integer variable ier. If A is judged 
to be positive definite, MCHOL will return a 0 in ier, and will return the factors 
L and D in the arrays L and D. Thus in addition to finding the decomposition 
if possible, MCHOL provides an easy check for positive definiteness of a matrix. 


A.1.2. The Gram-Schmidt Decomposition 


An important operation in many areas of mathematics and statistics is 
to find what is called an orthogonal basis for a matrix; that is, from one set 
of vectors, find a new set by some linear transformation such that the inner 
product of any two different new vectors is zero. We will use the Gram-Schmidt 
decomposition (see Clayton (1971)) to accomplish this aim. 


Theorem A.1.2 | GRAM-SCHMIDT DECOMPOSITION 


If X is an (n x p) matrix having full rank p, then there exists an (n x p) 
orthogonal matrix Q (that is, Q7Q is diagonal) and a unit upper triangular 
(p x p) matrix R such that 

X = QR. 


This decomposition of X is called its Gram-Schmidt decomposition (GSD). 
Note that X7X = R?Q?QR = R? VR where R? is unit lower triangular 
and V is diagonal with positive diagonal elements. Since the modified Cholesky 


decomposition X7X = LDL? of X7X is unique, we have that R? = L and 
Meal 


The GS Command 
If the array X contains the (n x p) matrix X, then the command 
GS(X,n,p,Q,R,ier) 


will attempt to find factors Q and R such that X = QR by what is called the 
modified Gram-Schmidt decomposition algorithm; that is, if we let Qo = X, 


and Q;,...,Qp = Q be a sequence of (n x p) matrices, and q;; denote the jth 
column of Q;, then GS calculates fori = 1,...,p—1: 


dy ata Aan 


aT ie er 
Ry = os Jit leer 
7] 


Qij = Gi-1,j3 — Rij ai-i,s, pouitl,...,p: 
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If |d;| < 10~?° for any i, then GS concludes that X is singular and returns a 
1 in the output integer variable ier. Otherwise, ier is returned as 0 and Q 
and R are returned in the arrays Q and R of length n*p and p*p. Note that 
the algorithm is done in place; that is, Qi,... ,Qp are actually calculated in a 
single matrix. See Problem C1.25 for an illustration of the relationship between 
MCHOL and GS. 


A.1.3. The Sweep Operator 


Many of the computational and theoretical results in regression, multivari- 
ate analysis, and time series can be expressed succinctly using what is called 
the sweep operator. 


Definition. Let A be an (n x n) matrix. The process of sweeping A on its 
kth diagonal element, denoted by B = SWEEP(k)A, is the process of forming 
the matrix B by (assuming Ax, # 0): 


1 
Bur = — 
kk ap 
Be ial deals (kth column) 
Akk 
12 fl ah, ike a (jth row) 
Akk 


Aik Ak; : ; 
Bg AG oe tfk,7 + k: 


If B, = SWEEP(k,)A, B, = SWEEP(k2)By, ..., B = SWEEP(k,)B,_1, we 


write B = SWEEP(k,...,k-)A, and say that B is the result of sweeping A 
on its diagonals k;,...,k,. 


The sweep operator has many useful properties as we see in the next 
theorem. 


Theorem A.1.3 | PROPERTIES OF THE SWEEP OPERATOR 


Let 


where B, C, D, and E are (r x r), (r x s), (s x r), and (s x s) respectively. 
Then 
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a) SWEEP(ky,...,kp)A can be found by sweeping the diagonals in any 
order, for example, 


SWEEP (ky, k2)A = SWEEP(ko, ki) A. 


b) SWEEP(k, k)A = A; that is, sweeping a second time on a given diag- 
onal undoes the effect of a previous sweeping on that diagonal. 


B-} B-'C 
c) SWEEP(1,...,r)A = 


: ; | if B is nonsingular. 
—-DB~ E-DB“’C 


B-—CE'!D -CE"} 


d) SWEEP(r+1,...,r+s)A= 
E-'D E7} 


if E is nonsin- 


gular. 


e) SWEEP(1,...,r+s)A =A} 


B-!+B-!C(E—DB-!C)-'DB-!_ —B-1C(E— DB-!C)-! 
‘d _(E— DB-'C)-!DB-! (E — DB-!c)-? 


if B, E, and (E — DB~'!C)~? are nonsingular. 


The sweep operator can be used to solve many theoretical and computa- 
tional problems. We consider two examples ofits use. First, the basic quantities 
for the regression of y on X (where X is (n x p)) can be found by applying 
property (c) (see Section A.5): 


XTX XTy + Var(A) B 
SWEEP(L,...7)| ' . IF o : 
y X yy —BT RSS 


To illustrate the theoretical utility of sweep, notice for the matrix A in 
Theorem A.1.3 that sweeping on all of its diagonals is the same as sweeping 
the matrix SWEEP(r + 1,...,7 + s)A on diagonals 1,...,r. Thus the upper 
left-hand corner in part (e) is the inverse of the upper left-hand corner in part 
(d) by part (c). Doing a similar calculation with +E~ replacing E in A gives 
the well-known matrix inversion lemma (see Rao (1973), p. 33 for example). 
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Theorem A.1.4 | THE MATRIX INVERSION LEMMA 


Let B and E be nonsingular (r x r) and (s x s) matrices and C and D be 
(r x s) and (s x r) matrices. Then 


(B+ CED)~' = B-'+ B-!C(E-!+ DB-!C)-!DB-!. 


We will use this formula extensively when we consider recursive regression 
in Section A.5, that is, when we consider adjusting estimators for the addition 
or deletion of observations. It can also be used to motivate the Kalman Filter 
Algorithm discussed in Section A.4. 


The SWEEP Command 


The SWEEP command has two forms. Suppose the array A contains the 
(n x n) matrix A. Then the command 


B=SWEEP(A,n,k1,k2,ier) 


attempts to form the matrix B by sweeping A on diagonals k1 through k2. 
The second form 


B=SWEEP(A,n, ind,m, ier) 


attempts to form B by sweeping A on the m diagonals whose indices are entered 
in the array ind. If a diagonal element less than 10725 is encountered, SWEEP 
will return a 1 in the output integer variable ier. Otherwise, it will return a 0 
in ier and the matrix B in the array B of length nen. 


A.2. Fourier Series and Spectral Representations 


The ideas of frequency domain analysis of data are an important part of 
time series analysis. We have attempted to motivate these ideas intuitively in 
the main part of this book. In this section we describe in more detail some 
of the mathematics behind the ideas. For more details the reader can consult 
Chapter 4 of Priestley (1981) or Chapter 7 of Anderson (1971). We recall that 
the basic idea of the frequency domain analysis of time series is to express 
various quantities as the “sum” of orthogonal sinusoids. 
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A.2.1. Fourier Series for Periodic Functions 


Suppose that f is a function defined on a finite interval [a,b]. By this we 
also mean to include functions that are periodic of period 6 — a. Throughout 
this book we have taken our interval to be [0,1], as if it is not, we can construct 
g(y) = f[(ze—a)/(b—a)] and apply the results to the function g. This interval is 
also natural for spectral densities as then frequency can be thought of as cycles 
per unit of time. Thus for monthly data, frequency 1/12 refers to possible cycles 
of length 12 months. The interval traditionally considered in mathematics is 
[—7, t], which means for time series that one has to perform mental arithmetic 
involving 7 in order to convert frequency to a physically meaningful quantity. 
Using the interval [0,1] also simplifies many of the formulas in the subject. 

Our aim is to decompose f into the sum of sinusoids that are independent 
in some sense. Thus we begin by trying to approximate f by a sum of sinusoids 
of periods 1,1/2,...,1/n for an arbitrary positive integer n. 


Theorem A.2.1 | FOURIER SERIES APPROXIMATION 


Let f be asquare integrable function defined on [0,1]; that is, fo f?(z)dz < 
co. Let 


9; (x) = a; cos 2njx + b; sin 2ajxz = C; cos(2njz — ¢;) 
be a sinusoid of period 1/j. Let 


falz) = > + Do g9s(2), 
g=1 


where do 1s some constant. Then the values of ap, a1,...,@, and b,...,6, that 
minimize : 
1 ‘i n 
0 
[ \f-F-Vae] @ 
Hl 


are given by 


1 
a; =2 | cos 2njz f(z) dz, fi Orlane 1h 


1 
by Sai} sin 2rjzx f(x) dz, ip esse (i 
0 


Thus the sum of sinusoids that is closest to f in this integrated square 
error sense (which is analogous to the sum of squares of errors in regression 
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analysis) has the coefficients given in the theorem. Note that f need not be 
square integrable in order to calculate these coefficients. In fact, f need only 
be absolutely integrable, that is, if |f(z) dz| < 00, since then for example 


1 1 
‘ If(2) cos2njzlde < | lf (z)|| cos 2rjz|dz 
) 0 


- 
< [ Is(e)lde <o, 


since |cos2mjz| < 1. Note that a square integrable function is absolutely 
integrable but not necessarily vice versa (consider f(x) = 1/\/z for example). 


Definition. Let f be an absolutely integrable function defined on [0,1], and 
let 


1 
a, =2[ cos 2njz f(z) dz, eS Vb tae 


1 
=? | sin 2njz f(z) dz, Fe NO 8. 
0 
and 3 
tae oe Sy [a; cos 2rjx + b; sin 2rjz]. 
2 ra 
Then the a’s and b’s are called the cosine and sine Fourier coefficients of hie 
while f, is called the nth partial sum Fourier approximant to fe 


In general, we will have that f # fy. To accomplish a harmonic analysis 
of f then, we hope that f, — f in some sense, and also that there is some 
sense in which the sinusoids 9), 92,... are orthogonal. In the next theorem we 
present results showing that in many cases we do have the convergence and 
orthogonality. 


Theorem A.2.2 | PROPERTIES OF FOURIER SERIES 


Let f be an absolutely integrable function defined on the interval [0,1], 
and let f,, f2,... be its approximating Fourier sums. Then 


a) The sinusoids g;(x) = a; cos 2njz +b; sin2njzx are orthogonal in the 


sense that 
1 0, j#k 
J ss(e)an(@) ae = 
0 a} oo BF, j= Ke 
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b) We have the following results about pointwise convergence of f,(x): 


i) At a point x where f is continuous and the left and right derivatives 
of f exist, 


Jim, tute eae): 


ii) At a point x where f is discontinuous but the left and right deriva- 
tives of f exist, 
: (et) Fe) 
tad al FS rane angio 
where f(t) and f(z~) denote the right and left limits of f at the point z; 


that is, the approximating Fourier sums converge to the average of the two 
values of f at the discontinuity. 


iii) If f is continuous and has square integrable derivative, then f, — 
f absolutely and uniformly. 


c) If f is square integrable, then 


lim | |f(z)— fn(z)|? dz = 0, 
0 


while 


1 2 00 
2 | f(a) dz = 2+) (az +82). 
0 k=1 


Implications: This theorem gives us our harmonic analysis for two types 
of functions—absolutely integrable (part (b)) and square integrable (part (c)). 
For square integrable functions, f, + f in the integrated squared error sense, 
which need not mean that f,(r) converges pointwise to f(z). Note that the 
second equation in part (c) (which is called Parseval’s identity) is analogous to 
a decomposition of “total sum of squares” in analysis of variance (the integral 
plays the role of the sum) into a sum of squares of amplitudes of the sinusoids 
for periods 1,1/2,1/3,.... For absolutely integrable functions, part (b,i) shows 
that under general conditions, f, (x) does converge to f(z) pointwise, while part 
(b,ii) explains the behavior of the Fourier approximants for the ideal bandpass 
filter example in Section 2.3. Thus at the two points of discontinuity in the 
transfer function, f,(z) converges to the average of 0 and 1, that is, to .5. 
Finally, part (a) shows that as long as the Fourier coefficients exist, then the 
sinusoids are orthogonal where now the inner product is defined as an integral 
rather than as a sum as it was in Section 1.4.3. 


SEC A.2 FOURIER SERIES AND SPECTRAL REPRESENTATIONS 367 


The Complex Form of Fourier Series 


Since e*? = cos@ + isin 0, we have 
2nijr —2nijr —1) (e272 _ e72tijr 
cos 2rjxz = peer See and sin2ajz = i=?) ( q 
2 2 
and thus 
a n 
ie a + )_ [aj cos Ixjx + b; sin 2njz] 
j=1 
n 
= re te 
j=—n 
where 
a; +b a a; — ib; 
es and Sk Aa pS apnea 7 Ole 
Note that 


1 
en | f(z)e?""77 dz. 
0 
Thus we have that f and the r’s are Fourier pairs: 


f(z) = 53 rpeae OE 


j=—00 

1 * 
y= if f(x)e?"" de. 

0 


The r;’s are called the complex Fourier coefficients of fe 
A.2.2. Spectral Representation of R 


In Section A.2.1 we were concerned with whether a function could be 
represented as a sum of sinusoids. In time series we have the converse problem; 
that is, we begin with the autocovariance sequence and ask whether there is a 
function having the R’s as its Fourier coefficients. The simplest, although not 
the most general, answer to this question is given in the following theorem. 
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Theorem A.2.3 | SPECTRAL REPRESENTATION OF R 


If {R(v),v € Z} is a positive definite sequence of numbers such that 
R(v) = R(—v) and )-y2._ ,, |R(v)| < 00, then there exists a continuous function 
f such that 


co 


a) f(w) = eS Roeser, w € (0, 1]. 


1 
b) R(v) = i f(w)e?"**" du, veEZ. 
) fw) = fl —w). 


This theorem gives a harmonic analysis of an absolutely summable auto- 
covariance function in terms of the spectral density function f. Unfortunately, 
the covariance function is not always absolutely summable, as illustrated by 
the harmonic process in Section 2.5.2. We can still obtain a harmonic analysis 
of a non-absolutely summable R as an integral, but the integral is of a special 
type called a Lebesgue-Stieltjes integral. In the spectral representation of R in 
Theorem 2.2.1, we wrote 


1 
R(v) =) cos 27uw dF'(w), 
fn) 


where the integral is of the Lebesgue-Stieltjes type. Instead of defining this 
type of integral (which is rather complicated), we describe in the next theorem 
a way to calculate and interpret it in the special cases with which we are 
concerned. 


Theorem A.2.4 | LEBESGUE-STIELTJES INTEGRAL 


Let g and F be bounded functions defined on [0,1]and for a positive integer 
m., consider partitioning [0,1] into the 2™ intervals 


1 ln eZ Cn 
[Oa] fam ae] [Fal 


Thus if we let Wm; = 7/2”, forj =0,1,...,2™, the kth interval is from Wi hoi 
to Wm, fork =1,...,2™. Now let 


om 
In = S59 (wn,k) [F (Wm,k) 7 F(Wmk-1)] ; 
sh 
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where Wink = (Wm,k-1 + Wm,k)/2. Then if g is Lebesgue-Stieltjes integrable 
with respect to F, and g is continuous (and therefore is also Riemann-Stieltjes 
integrable), we have that limm_—oo Im exists and is finite, and 


1 
| g(z)dF(z) = Jim In. 


From this result we have that 


1 25 
— —_— j / —_— 
R(v) = i} cos 27uw dF (w) = lim > cos 2mvws, 4 LF (Wm,k) — F(Wm,k-1)] ; 


which means that the autocovariance sequence R can be regarded as the sum 
of a large number of cosines with amplitudes given by “increments” of the 
spectral distribution function F. Thus if there is a large increase in F be- 
tween frequencies Wm ,—1 and Wm,k, the sinusoid having frequency w! , will be 
prominent in the decomposition of R. In fact, if the only large increase in F is 
in the interval containing Win, ,, then R will appear almost exactly sinusoidal. 
The logical limit of this idea is the autocovariance of the harmonic process. 


The Harmonic Process 


Consider the harmonic process having a single frequency component of 
frequency wo € (0, .5); that is, R(v) = o? cos 2muwo. Define 


0, 0<w <u 


2 
F(w) = a wo <w < 1l—wy 


07, l-wo<w<l, 
and let w, be the center of the interval in the partition of [0,1] at the mth 
step of the limiting process. If wo is on the boundary between two intervals, 


we choose w,,, to be the center of the one containing the jump in F. Thus note 
that 1—w,, is the center of the interval containing 1 — wo, and we have 


tes 
lim cos 2ruw), 4 [F (wm,k) — F (wm,k—1)] 
: o? o? 
= lim, E COS 27 VW, + 7 098 2nv(1 — vm) 


= 0" cos 27 VW, 
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since Wm — Wo and cos2muw = cos2mv(1—w). Thus this definition of the 
spectral distribution function does indeed lead to the correct autocovariance 
function. We note that in the general theory of representing sequences, the 
function F is unique, and thus our definition of F is the only one that represents 
i: 

In general, if F is absolutely continuous, we can see that the Lebesgue- 
Stieltjes integral reduces to the usual Riemann integral since then 


: tte FWint) =F (Gees) 
/ cos 27ruw dF(w) Ss im. om Me QrvWwin k Jae ee] 


om Lon 
. = » GF (Wing) 
= lim — ) cos2rvuw,, , ————— 
m-—roo 2™ : dw 
k=1 
1 
= | cos 2ruw f (w) dw 
) 


by the definition of the Riemann integral. 
A.2.3. Spectral Representation of X 


It is possible to obtain a harmonic analysis of the time series X itself (see 
part (d) of Theorem 2.2.1). The representation is different than that for R in 
three ways. First, the representation for R is in terms of cosines only. This is 
because R is an even function, that is, R(v) = R(—v). Second, the increments 
F(w,) — F(w;) are replaced by the increments of two uncorrelated stochastic 
processes (see below). Finally, since the representation will be as a limit of 
sums of linear combinations of random variables, the usual limit cannot be 
used, rather the limit is a limit in mean square (see Section A.4). 

A stochastic process {C(w),w € [0,1]} is an indexed collection of ran- 
dom variables. The random variable C(w2) — C(wi) for 0 < w1 < we < lis 
called an increment of C. If increments defined on nonoverlapping intervals 
are uncorrelated, we say that C has uncorrelated increments. 

Thus the spectral representation of X means essentially that 


Q™ 


X(t) Lim. > [eos Qwik [C(Wm,k) — C(wm,k-1)] 


k=1 
+ sin 2TVW ink [S(wm,k) — S(wm,r-)]| ) 


where l|.i.m. denotes limit in mean square. This again allows us to think of a 
particular infinitely long realization from X as the sum of sinusoids whose am- 
plitudes have been chosen according to some random mechanism (determined 
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by the processes C and S) whose properties (such as variance) are described 
by the spectral distribution function F. 


A.3. The Fast Fourier Transform 


Many of the calculations in time series analysis require the calculation 
of the discrete Fourier transform of a set of numbers (see Section 1.4.3 for 
example). In this section we describe the fast Fourier transform. The basic 
ideas were developed and popularized by Cooley and Tukey (1965) and Gen- 
tleman and Sande (1966). The FFT is a classic example of the use of recursion 
to greatly reduce the number of operations required to solve a computational 
problem. Although there are a wide variety of algorithms known as fast Fourier 
transforms, almost all of them have two essential elements: (1) a recursion that 
allows one to find the DFT of n points as a simple combination of DFT’s for 
subsets of the points, and (2) a reordering of the original data so as to minimize 
the number of complex exponentials that need to be calculated and/or stored 
while the recursion is being performed. 


A.3.1. The FFT Recursion 
Let W,, = e~?*/". Then we can write the DFT of X(1),...,X(n) as 


L(k) 27 XC) WE MEDS Wo kk dg 


t=1 


Theorem A.3.1 | THE FFT REcurRSION 


Suppose n can be factored into n = rs. Define the (r x s) matrix V to 
have jth column 
X(j),X(s+J),...,X((r—1)s + J), se Oa as 
Then : 
Z(k) =) W-Y"-Y Z,(mod(k — 1,r) +1), 
1=1 
where Z;(1),...,Z(r) is the DFT of the Ith column of V. 


Proof: We can get Z(k) by adding over the elements of V: 


Z(k) = S> Sy Vin WEED) o4 0-2) 
A 


i=l im= 


Es She Wk-1)(-1) De Vint W(k-1)(m—1)s 


fat m=1 
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since Vy = X((m—1)s +1). But s/n = 1/r which gives WZ = W,. Further, 


k snl feo i 


k-1=[ : —*|r + mod(k — 1,7), 


—"}r-+(k-1)-[ 


and thus 


wre 1)(m—1)s _ wir 1){[ t= +]r+mod(k—1,r)} 


ca Wie Decds ia: 
(e=*)r ; 
since W; " ° = 1 because e~2** = 1. Thus 
$s r 
Z(k) = Ss, WDE) y3) Vex W,(rn—1)mod(k—1,r) 


l=1 ina 


= 5° w- 9 Z (mod(k — 1, r) + 1). 
S12 


Thus it takes sr? = nr operations to calculate the DFT’s of all s columns 
of V and then ns operations to calculate all n Z’s, making a total of nr+ns = 
n(r +s) operations instead of n?. Further, if r is itself factorable, we can use 
the same idea to more rapidly find the DFT’s of the columns of V. This process 
can be continued until all of the prime factors of n are exhausted, giving a total 
of n(p; +---+ px) operations where pi,...,pK are the prime factors of n. 


A.3.2. Reordering Data 


The recursion given in Theorem A.3.1 can be further improved if we can 
order the calculations in such a way that all of the ones using a certain complex 
exponential can be done at the same time. To illustrate this, consider the case 
of n=8 and let s=2: 


Z(3) = Z1(3) + W2Z2(3 
Z(4) = 2, (4) + WeZ2(4 
Z(5) = Z1(1) + WeZo(1) 
Z(6) = Z(2) + We Z2(2) 
Z(7) = Z1(3) + Wg Z2(3) 
Z(8) = Z1(4) + Wg Z2(4) 
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where Z, is the DFT of the odd numbered X’s and Z2 is the DFT of the even 
numbered X’s. Thus 


4,(1) = Z(1)+ WeZr2(1) — Za(1) = Zar (1) + W2Z29(1) 
41(2) = Z12(2) + W4Z12(2) Za (2) = Zo1(2) + Wi Z29(2) 
4,(3) = Z11(1) + WiZ2(1) Za (3) = Zor (1) + W2Zo9(1) 
4, (4) = Z12(2) + WeZ12(2) 224) = Zor (2) + W3Z29(2) 
where 
41(1) = X(1) + W2X(5) 
) = X(1) + Wz X(5) 
Zi2(1) = X(3) + W2X(7) 
) 


221 he X (2) + we 


( 

( ( 
2Z12(2) = X(3) + Wi X(7 

( 6) 

( 


X( 
Zi (2) = X(2) + Wy X (6) 
Zo2(1) = X(4) + W)X(8) 
Z2(2) = X(4) + W7X(8). 


Thus if we start by permuting the X’s we can summarize the calculations by 


X (1) Z1(1) 21 (1) Z(1) 

X (5) Z12(1) Z2(1) Z(2) 

X (3) Z21(1) 2, (2) Z(3) 
Aaa Z22(1 Z2(2) ( 
x0) ¥ 7) * Z;(3) zi z(5) 
X (6) Z32(2) et Z (6) 

X (4) Zi (2) Z1(4) Z(7) 

X (8) 222(2) Z2(4) Z(8) 


where the first arrow means first combine all four consecutive pairs using W? 
and then again using W}. The second arrow means combine the first two pairs 
using W?, the next two using Wj, the next two using W#?, and the last two 
using W3. In the last step we maiibane the first pair using Wg, the next pair 
using W2, and so on. Thus at each step we are fixing the complex exponential 
and then doing all combinations using that value. 

To determine the index k of X(j) in the permuted array, one need only 
write down the binary representation of j—1, reverse the bits, and then add one. 
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For example, X(5) is mapped to position 2 since 5-1=4=100-4001-4010=2. 
Note that if X(j) gets mapped to position k, then X (k) gets mapped to position 
j: 

There are many variations on this theme and further simplifications that 
can be made. In TIMESLAB we use the algorithm described by Singleton 
(1969) which is in some ways still the standard by which newer methods are 
judged. We chose this algorithm because it performs the transform in place 
and will work for a wide variety of different prime factors, although as we saw 
in Section 1.4 it does require that n not have any large prime factors. 


A.4. Random Vectors and the Multivariate Normal Distribution 
In elementary statistics we often avoid treating vectors of random variables 

since the variables are independent and can be treated separately. In time series 

analysis this is usually not the case. 

A.4.1. Basic Definitions 


We begin our discussion with a series of definitions. 


1. A d-dimensional random vector X is a vector of d random variables, 
that is, X = (X,,...,Xq4)7, where X,,...,Xq are random variables. 


2. The joint cumulative distribution function (cdf) Fy of the random 
vector X is given by 


Fx (x) = Py (crea) = Pr(Xi SS Rly ob op BGS ave 


3. If there exists a function fx of d variables such that 


F(x) = ff feoday, 


then X is said to be a continuous random vector and fx is called the joint 
probability density function (pdf) of X. 


4. If Y and X are r-dimensional and d-dimensional random vectors, then 


Z = (¥7,X7)? is an (r+ d)-dimensional random vector. If fz, fy, and fx 
are the pdf’s of Z, Y, and X respectively, then the functions 


fy\x = B and fxjy = i 
x 


fy 


are called the conditional pdf’s of Y given X and X given Y, respectively. 
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5. The mean vector x and covariance matrix Ex are given by 
E(X, 


) 
yx = B(X) = ats = [+ | xtx(oax 


Ex = Var(X) = E[(X — px)(X — px)7] 


t. ii i (x — px) (x — wx)” fx (x)dx 


Var(X1) Cov(X1,X2) Cov(X1,X3) ... Cov(X1, Xa) 
Var(X2) Cov(X2,X3) ... Cov(X2, Xq) 
= Var(X3) 
Symmetric Var(X4) 


If E(X) = px and Var(X) = Dx we will write 


The covariance matrix Lxy of X and Y is the (d x r) matrix whose (j, k)th 
element is Cov(X;,Y,). The conditional mean Hy|x and variance Ly;x of a 
random vector Y given X are defined in the same way as py and Ly, using 
the conditional pdf of Y given X. 


6. If we partition the (r + d)-dimensional random vector Z into the r- 
dimensional and d-dimensional random vectors Y and X consisting of the first 
r and last d elements of Z, then we can partition Bz and Xz similarly as: 


Y Lyy Lyx 
and Yz= ) 


XLxy Uxx 
where in fact, 
Lyy = Var(Y), Xxx = Var(X), 
Lyx =Cov(Y,X),  LExy =Cov(X, Y) = E7,. 
7. The r-dimensional random vector Y is called a linear transformation 


of the d-dimensional random vector X if Y = AX + b where A is an (r x d) 
matrix of constants and b is an r-dimensional vector of constants. 
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8. If X is a d-dimensional random vector and A is a (d x d) matrix of 
constants, then the scalar random variable Q = X7 AX is called a quadratic 
form in X and A is called the matrix of the quadratic form. 


A.4.2. Basic Facts 


With these definitions in mind, we next state some theoretical results that 
are used throughout the book. We begin with the following theorem. 


Theorem A.4.1 | MEAN AND VARIANCE OF A LINEAR FUNCTION 


If X ~ (px, Xx) then 
Y =AX+b~ (Apx + b, ADxA?’), 
that is, 


E(AX + b) = AE(X) +b 


Var(AX +b) = AVar(X)A7. 


Most of the theory and methods of time series analysis are built upon the 
properties of the multivariate normal distribution. 


Definition. The d-dimensional random vector X having mean px and vari- 
ance Xx is said to have the d-dimensional normal distribution if its pdf is given 
by 


fx (x) = (20) 4/?|Ex|-1/? exp (ae — px) Z51(x — px) 


for x € R4, that is, in d-dimensional Euclidean space. Such a random vector 
is denoted by X ~ Na(ptx, Ex). 


The important properties of the multivariate normal distribution are sum- 
marized in Theorem A.4.2 (see Rao (1973), Chapter 8 for example). 
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Theorem A.4.2 | PROPERTIES OF THE MULTIVARIATE NORMAL 
Y By Lyy Lyx 
the ~ Nr+d ) ) 
x px Lxy Uxx 


a) The characteristic function of Z is given by 


then 


; 1 
z(t) = E(e'*"2) = exp(it? wz — 5t Ext). 
b) If A is an s x (r + d) matrix of constants and b is an s-dimensional 
vector of constants, then 
AZ+b~N,(AZ+b,ADzA‘*); 


that is, linear transformations of normal random vectors are also normally 
distributed. 


c) i) The conditional distribution of Y given X is 
Y|X ~ N, (py + Lyx By (XK — wx) Lyy — Lyx Ly =xy); 


that is, the pdf of Y given X is that of an r-dimensional normal random vector 
with mean and variance given by 


By|x = E(Y|X) = py + Lyx Dy (X — px) 
Ly\x = Var(Y|X) = Dyy — Lyx Ly Exy; 


that is, the conditional mean of Y given X = x is a linear function of x, while 
the conditional variance is the same no matter what x is. 


ii) Conversely, if 
Y|X ~ N, (uy + Ly xByy(X — wx), Lyy — Lyx Dx, Exy) 
and X ~ N,(px,2xx), then 


Y By Lyy Lyx 
A= ™~ INeid ) : 
x Bx Xxy <Uxx 


d) If A is a (d x d) nonsingular matrix of constants, then the distribution 
of Y|X and Y|AX are the same. 
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e) Cov(AX, BY) = AL xyB7. 
f) The quadratic form 
Q = (XK — px)" Ex (X — ax) 
has the chi-square distribution with d degrees of freedom, and we write Q ~ x3. 
g) The quadratic form 
Qa = (KX — px)? A(X — px) 


has the same distribution as the random variable Dee AjZj; where 21,...;Zs 
are independent x? random variables, and the ’s are the nonzero eigenvalues of 
Axx. This reduces to the result of part (f) if A = Dy since the eigenvalues 
of the d-dimensional identity matrix are just one repeated d times, and the sum 
of d independent x? random variables is x%. 


h) The mean and variance of a quadratic form in normal variables are 
given by 


E(X7 AX) = tr(ADxx) + pe Apx 
Var(X7 AX) = 2tr[(ADxx)?] + 4ux AD xpx. 
i) The two quadratic forms Q; = X7 AX and Q2 = X7 BX are indepen- 
dent if and only if ALx xB? = 0. 


j) If Qi and Q2 are two independent x? random variables having n; and 
nz degrees of freedom, then the random variable 


— Qi/n1 
ay Q2/n2 


has the F distribution with n, and nz degrees of freedom and we write F ~ 
Pista: 


k) If Z ~ N,(p, 07) and Q ~ x? are independent, then the random variable 


(Z — p)/o 


es) SS 


VQ/v 


has the Student-t distribution with v degrees of freedom and we write t ~ t,. 
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The DIST Command 

The pdf, cdf, and quantile functions for the N(0,1), ty, May Bt ho, ep 
distributions can be evaluated via the DIST command, as can the general in- 
complete gamma function. 


Simultaneous Confidence Intervals 


A result that is useful in finding simultaneous confidence bands for func- 
tions is given in the next theorem (see Scheffé (1959), p. 406). 


Theorem A.4.3 | SIMULTANEOUS CONFIDENCE INTERVALS 


Let X be an arbitrary p-dimensional random vector and let V be a Positive 
definite (p x p) matrix of constants. Then the p-dimensional vector a satisfies 


(X — a)? V(X -—a) <1 


if and only if 
n” (X —a)| < (h7 V-"h)-*/? = D 


for all p-dimensional vectors h; that is, 


fox) ha < he XD 


A.4.3. Prediction of a Random Vector 


Consider random vectors Y and X such that Y has d elements, X has r 


elements, and 
Y Bx 
E = 
(x)= [er] 
(x) a a 
Var y= : 
Yxy <Uxx 


We consider two criteria for predicting Y from X: (1) best unbiased prediction 
and (2) best linear unbiased prediction. 
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Definition. The best unbiased predictor of Y from X is that function Y of 
X that has the same expectation as Y and is closest to Y in expected mean 
square error; that is, 


B[(¥ -¥*)(¥-¥*)"] -B[(¥-¥y(v¥-¥)?| 


is positive semidefinite for any other unbiased function Y* of X. The best 
linear unbiased predictor Y of Y given X is that linear function of X having 
the same expectation as Y and is closest to Y in expected mean square error, 
that is, Y = AX where 


B[(Y — A*X)(¥ - A*X)7] -E|(¥ — Axy(y - Ax)" | 


1s positive semidefinite for any other linear function A* X that is also unbiased 
for Y. 


It will turn out that Y is the conditional expectation of Y given X. Con- 
ditional expectation is a crucial concept in time series analysis and can be 
defined at many levels of generality. A more general and useful definition of 
conditional expectation than that given in Definition 5 in Section A.4.1 is given 
by the following (see Parzen (1962b), for example). 


Definition. Let Y be a random vector having finite mean. Then the condi- 
tional expectation E(Y|X) of Y given X is that function ¢ of X such that 


E([¥ - 4(X)]g(X)) =0 


for any bounded function g of X. 


This definition is called descriptive (as opposed to the constructive defini- 
tion in Section A.4.1) as it says that the “residual” Y—E(Y|X) is uncorrelated 
with any other function of X. In this sense there is no more information about 
Y in X after having removed E(Y|X). We note that the conditional vari- 
ance Var(Y|X) of Y given X is just the conditional expectation of the random 
variable [(Y — E(Y|X))(Y — E(Y|X))7] given X. 

Letting g(X) = 1 in this definition yields E(¢(X))= E(Y), and thus 


Cov(Y — 4(X),g(X)) = 0. 


With these definitions in mind, we have the following theorem. For simplicity 
we assume that the random vectors involved have zero mean. 
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Theorem A.4.4 | GENERAL PREDICTION THEORY 


Let Y and X be jointly distributed random vectors having zero mean. 
Then 


a) The best unbiased predictor Y of Y given X and its error variance 
Ly = Var(Y — Y) are given by 


Y=E(Y|X) and Yy = Var(Y|X). 
b) If &xx is nonsingular, then the best linear unbiased predictor Y of Y 
given X and its error variance Dy = Var(Y — Y) are given by 
Y = AX and Ly =Dyyi Lyx Lyx Uxy, 
where A satisfies the prediction normal equations 


DxxA? = Exy. 


c) If¥ and X are jointly multivariate normal, then Y = Y and diy = De. 


Proof: To prove part (a), let Y* be an arbitrary function of X having mean 
0, and let 


6=Y-E(Y|X), y=E(Y|X)-Y*,. a=Y-Y*=8+4. 
We need to show that E(aa7) — E(55°) is positive semidefinite. We have 
E(aa’) — E(857) = E [(6 + )(6 + 4)7] — E(667) 
= E(y57) + E(5y") + E(y7"), 


the third term of which is positive semidefinite since for any vector h not 
identically zero, we have 


h* E(y77 )h = E(h? y77h) = E [(h7y)?] > 0. 
Now E(y) = E(6) = 0 and thus E(7é7) = Cov(y,6) and E(éy7) = Cov(6, 7). 


But both of these covariances are zero since -y = E(Y|X) — Y* is a function of 
X since E(Y|X) and Y* are both functions of X. 
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To prove part (b), let A be an arbitrary (r x d) matrix. Then 
S(A) = E[(¥Y — AX)(Y — AX)7] 

= E(YY7) — AE(XY7) — E(YX7)A? + AE(XX7)A7 

= Dyy — AExy — LyxA? + ALDxxA7 

= (A — Ey xByy)Exx(A — Lyx Bx)” 

+ Dyy — Lyx LhzyUxy. 
Thus Ly = S(A) = Yyy — LyxLxxUxy and 

S(A*) — S(A) = (A* — Eyx2xh)Exx(A* — Lyx Exh)” 


which is clearly positive semidefinite for any matrix A*. 
Finally, part (c) follows from part (c) of Theorem A.4.2. 


The Kalman Filter Algorithm 


If we have a sequence of random vectors that are multivariate normal and 
satisfy a recursive relationship of a particular form, then predictors can also 
be found recursively. This type of recursion was used extensively by Kalman 
(1960) and thus the resulting algorithm is usually referred to as the Kalman 
Filter Algorithm. 


Theorem A.4.5 | THE KALMAN FILTER 


Let Xo, Xi,... and Z, Z2,... be sequences of r- and d-dimensional ran- 
dom vectors satisfying the two equations (called the state and observation 
equations respectively) for t > 1: 

Xt = AtXt-1 + we 
Z, = HeX: + ve, 
where the A; and H; are full-rank (r x r) and (d x r) matrices of constants, 
wt ~ N,(0,, We) 
Vine Na(Oa, Ve), 


all of the w;’s and v¢’s are independent, and 


Xo ~ Nr(Ho,To). 
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Let % and %; denote the mean and variance of the conditional distribution of 
X; given Z;,...,Z,. Then 


a) Fort > 1, 
X, = ArX;_1 + R,H?(H,R,H? + V;)~1e, 
2; = R; — R,H? (H;R,H? + V;,)-"H,R:, 
where 
Ry = A;34_1A7 + W; 
e, = Z, — Hi ArX:-1, 
and 


Soe Do, Xo = po. 


b) Fort > 1 andh > 0, let erie and Seth—ife-i denote the mean 


and variance of X¢4n-1 given Z1,...,Z¢—1. Then 
fe Xe h => 0 
Xepa—1e-1 = . 
Att¢h—-1Xt4n-2]t-1, h>1 


Yr-1, h=0 


Le4h-1\t-1 = es fe 
Ac¢n-12¢4h-2)t-1 At 4-1 teWeencay La Dal: 


Proof: In our proof, we follow Meinhold and Singpurwalla (1983). Let Z 
represent the set of variables {Z1,...,Z:}. We know that the distribution of 
X,|Z', which is what we want, is normal since Xo and all of the w;’s and 
vt’s are normal, and thus all of the X;’s and Z;’s are jointly normal. We 
want to show that the mean and variance of X;|Z' satisfy the relations in 
part (a). Note that X,_, = E(X;_;|Z*-+) is a linear function of Z*-! and 
thus e, = Z; — H,A;X;_, and Z‘-! are a linear transformation of Z‘. Thus 
by part (d) of Theorem A.4.2, we know that the conditional distribution of 
X,|Z* is the same as that of X,le,,Z'-!. The distribution of Xzle¢, Z*-1 can 


be found from that of ie |Z‘-? by conditioning on e; (see part (c,i) of 


et 

Theorem A.4.2), and the distribution of (ais) |Z‘~? can be found from those 
t 

of X;|Z‘~* and e,|X;,Z'-1 by part (c,ii) of Theorem A.4.2. When t = 1 we 
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want the distribution of X|Zi, so we argue in the same way except there is 
no Zt. 
Since X; = A;Xz-1 + wi, we have 
E(X_|Z'-1) = AcE(X+-1|Z°!) + E(we|Z**) 
= A;X¢-1 
Var(X;|Z°-1) = AtVar(Xe-1|Z*-!) A? + Var(w;|Z‘-*) 
= A,X-1A7 + We, 


that is, 
X,|Z'-1 ~ N,(AtXe-1, AtX4-1 AP +W;). 


Since e = Zt — HA xe. and Zt = HX; + Vi, we have e = H;[X; = 
AtX¢_-1] + vi, and since X;_1 is a function of Z'-1, we have 


e,|X,, Z°-? ~ Na(He(Xt — AeXe-1), Vt). 


Let Ry = A;Sy-1 AT + W,, and to use part (c,ii) of Theorem A.4.2, make the 
identifications 


Seep Vi ee pyrtA, Ser oreo) 
Yxx © H,R,H? + Vz, Xxx + H:R:, Lyy © R,. 


To verify that this identification is valid, we suppress reference to Z'~1 and 
note that 


e:|X; o X|/Y 
~ Na(ux + DxyDyt(¥ — py), Bxx — UxyLyy Lyx) 
= Nqa(H;R;R;+(X; — ArX¢-1), H:R:H? + V; — H:R:R;'R,H7) 
= Na(H:(Xt — AtXe-1), Vz) 


as required. Thus we have by part (c,ii) of Theorem A.4.2, 


psi izt-} (| 0 | eae +V; H;R; 
™ INrtd n ) ) 
er YH wre R,H? R, 


which in turn gives the desired recursion for X;, and ¥; by part (c,i) of Theorem 
A.4.2. 
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Part (b) for h = 0 follows by definition, while for h > 0 the results come 
from the fact that Xi4,_1 = Attn—2Xt4n—2 + Wey n-1- 


Alternate Expression of KFA 


There are many algebraically equivalent ways to express the Kalman Filter 
Algorithm (KFA). In the theorem we wrote it in a way making it clear how the 
recursion goes from one step to the next, and how it starts. To obtain more 
intuitively appealing expressions, define 


X;=E(X,|Z'-1) and ¥, = Var(X,|Z'-1), 


that is, the mean and variance of X; having only observed Z up to time t — 1. 
Then clearly, 


xX? = Ae and zs: hte 


that is, our best “estimate” of X; without observing Z; is A¢Xz_}. If we define 
the Kalman gain matrix K; by 


K; = ©, H? (H;5;H? +V;)7?, 


we can write 

X; = X,+ Ki(Z - H;X;); 
that is, our best guess of X; given Z' is a linear combination of X, (our best 
guess of X; given Z‘~') and a variable expressing how well H;X; predicts 
the new observation Z;. Note how similar this is to recursive regression (see 


Theorem A.5.5). We can summarize the alternative expression for the KFA as 
Xo = E(Xo), Xo = Var(Xo), and 


KS K Ke, 
Y; = [I — KH], 
where 
Xp A,X 
e: = Z; — HX; 
¥, = A,Sy_1 Al + W; 


K; = o;H? (H,5,H? ae Vii-% 
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A.4.4. Convergence of Random Variables 
Many of the theoretical results for statistics calculated in time series anal- 


ysis are stated as limiting results as the length of the time series gets large. In 
this section we discuss the various modes of convergence of random variables. 


Definition. Let {X,,n = 1,2,...} be a sequence of random variables such 
that the cdf of X, is F,. Let X be another random variable, and suppose that 
the cdf of X is F. Then 


a) Xn converges to the constant c IN PROBABILITY if Ve > 0, 
lim Pr (|X, —c| > €) = 0. 
n—oco 
We denote this by Xy, aes c, and say that X, eX if Xy — X =): 


b) X, converges to the constant c WITH PROBABILITY ONE (or almost 
surely) if Ve > 0, 


lim Pr (sup |X, —c|> c} =O; 
N-0o n>N 
We denote this by X, —+ c, and say that X, —+ X if X, — X 40. 
c) X, converges to X IN MEAN SQUARE if 
G ie 2 = 
jim E (|Xn Xe 0: 
We denote this by X, —> X. 
d) X, converges to X IN DISTRIBUTION if 
tim F,, (2) =.F(2) 
for every x where F is continuous. We denote this by X, aS 


e) If {Z,,n = 0,+1,+2,...} is a doubly infinite sequence of random vari- 
ables, and {@;,j7 = 0,41,+2,...} is a sequence of constants, we say that 


as a limit in mean square if 


M 
Ynjm = ye B3Zn-j = Yn. 
j=-M 


SEC A.5 LEAST SQUARES REGRESSION ANALYSIS 387 


We note that a statistic 6,,, calculated from a realization of length n, is said 
to be a weakly (strongly) consistent estimator of a a parameter 6 if 6,, > @ 
(6, ast 9). Also, a random variable X,, is said to be asymptotically normally 
distributed with asymptotic mean Hn and asymptotic variance o?, if 


Xn = Hn oeX NO. 1). 


We denote such a random variable by X, ~ AN (4n, 02). For example, the 
usual Central Limit Theorem of statistics says that X ~ AN (u,07/n) 


Convergence of Random Vectors 


For a sequence of random vectors {Xn}, we can define concepts of con- 
vergence analogous to those for a sequence of single random variables. Con- 
vergences in probability, with probability one, and in mean square are defined 
pointwise; that is, we say that X,, ate XW Sob Kcor X, = X if each 
element of X, converges to the corresponding element of X. For a random 
vector to be asymptotically normal we will write X, ~ AN, (Hn, Zn) if Z, has 
nonzero diagonal elements for all n sufficiently large, and if for every vector h 
such that h?,h > 0 for all n sufficiently large, then 


h?X, ~ AN(h7 py, h7D,h). 


Thus a random vector is said to be asymptotically normal if all linear functions 
of it are asymptotically normal in the scalar sense. 

Several times in the book (see Problem T3.3 for example), we make use 
of the fact that a continuous function of an asymptotically normal sequence of 
random vectors is also asymptotically normal. We state this formally in the 
following theorem (see Serfling (1980), p. 122, for example). 


Theorem A.4.6 | CONTINUOUS FUNCTION THEOREM 


Suppose that X, ~ AN, (p#,6,%), where £ is a covariance matrix and 
b, +0 asn-—- oo. Let 


(x)= (91(x),---, 9p(x))*, xe (zi78. 1525)" 5 


be a vector-valued function for which each component function g;(x) is real 
valued and totally differentiable. Let G be the (p x k) matrix whose (i,j) th 
element is the partial derivative of g; with respect to x;, evaluated at the vector 
pb. Then 

8(Xn) ~ AN(g(u),b; GEG"). 


A.5. Least Squares Regression Analysis 
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Regression analysis plays two important roles in the study of time series. 
First, regression is an important tool in the transformation of a time series 
to remove trends, cycles, or other deterministic functions from data. Just as 
importantly, much of the theory and methods of time series analysis can be 
seen as being analogous to the theory and methods of regression analysis. Thus 
in this section we will try to carefully describe many of the ideas of regression 
analysis. 


A.5.1. The General Linear Model 


In a wide variety of statistical settings one is presented with n observations 
Y1,---)Yn On some dependent variable y, and corresponding to y; there is a vec- 
MOF es =e , Xip) of observations on independent variables Xj,..., Xp. 
Usually one seeks to determine models which adequately express the average 
behavior of y as a linear function of the X’s. We will consider the values of 
the independent variables to be fixed, that is, nonrandom. If they are in fact 
random, the material that we will discuss in this section will be considered to 
be conditional on the observed values of the independent variables. 

Let y = (y1,--.,; Yn)’ be an n-dimensional random vector satisfying the 
general linear model 

y=XfPrte, 


where X is an (n x p) matrix of fixed constants, B is a (p x 1) vector of 
unknown parameters, and € is an (n x 1) random vector satisfying E(€) = 0,, 
Var(€) = o7I,. We will assume that X is of full rank p, that is, n > p, and 
no column of X is a linear combination of the rest, that is, Xh # 0, for any 
(p x 1) vector h that is not identically zero. This means that X7X is also of 
full rank p and is in fact positive definite; that is, if h # 0,, then 


h? X?Xh = (Xh)?Xh=27z > 0, 


since z = Xh F Op. 
Thus let B be the unique vector satisfying the prediction normal equations 
XTX = X7y, that is, 


B = (X?X)-!X7y = Ay. 


Thus f is a linear function of the data y and A = (X7X)—!XT is the matrix 
of the linear transformation. 

Let y = XB = X(X7X)-!X7y = Hy denote the vector of fitted (or 
predicted) values, and let e = y—y = (I, —H)y denote the vector of residuals. 
Note that H is called the hat matrix and that it is idempotent, that is, H? = H 
since 


H? = X(X*X)— XS eek SIX RX) OX = 


Also H and (I, — H) are symmetric while (I, — H) is idempotent. 
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Define the residual sum of squares (RSS) by 


RSS =e"e=y" (I, — H")(I, — H)y 
= y’ (I, — H)*y = y7 (In — H)y 


=y"y—y’Hy =y’y-y'y 


=y’y—fTXx’y, 
and let 
RSS 
CaS r, 
n—p 


The model y = Xf +€ can be thought of as a decomposition of y into 
the sum of two parts, the first being deterministic and the second being ran- 
dom. The deterministic part is something that can be explained or controlled 
scientifically while the random part is unexplainable and uncontrollable. It is 
important scientifically to get an idea of how much of the variability in y is 
due to each of the two parts of the model. If X contains a column of ones, that 
is, the model has an intercept term, then one usually measures the propor- 
tion of the variability due to the deterministic part by the multiple correlation 
coefficient 


where s? and of are the sample variances of e and y respectively. 


To measure the relative importance of a subset of the independent variables 
in explaining the variability in y, one traditionally finds RSS for the model 
with and without the columns of X corresponding to the variables in question 
and then judges whether including the “extra” columns leads to a “significant” 
reduction in RSS. We can phrase this also in terms of the coefficients of 
the independent variables. The variability in y “explained” by B2 above and 
beyond that “explained” by f is measured by 


R(B2|B1) = RSS(B1,B2) — RSS(B,), 


where £2 denotes the subset of B being examined, 8, represents the other 
elements of 8, and the notation RSS$(@) denotes the residual sum of squares 
for the model containing only the independent variables corresponding to the 
elements of a vector 6. 
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A.5.2. Results for the General Linear Model 


We next summarize the basic properties of the quantities defined above 
without making any further assumptions about e. 


Theorem A.5.1 | PROPERTIES OF LEAST SQUARES ESTIMATORS 


Under the conditions described above, we have 


a) B minimizes 
S(B) = (y — XB)" (y - XA); 


that is, B is that value of B that minimizes the sum of squares of vertical 
distances between the elements of y and Xf. 


b) E(B) =, — Var(B) = 0?(X7X)-}. 


c) In the class of all unbiased estimators of B that are linear functions of 
y, B is best in terms of variance; that is, if B is any other linear function of y 
having expectation B, then Var(f) — Var(f) is positive semidefinite, that is, 


h7[Var(8) — Var(A)]h > 0 
for any h $ 0p. 


d) The fitted value y; and residual e; are uncorrelated, that is, Cov(9;, e;) = 
0. Thus we can write an orthogonal decomposition of y by 


y=y+te, Cov(y,e) = 0, 
and 
Var(y) = Var(Hy) = o?H? = o?H 
Var(e) = Var([I, — H]y) = o?(I, — H)? = o?(I, — H) 


Var(y) = Var(#) + Var(e) = 07H + 0°[l, — H] = 07Iy. 


e) &? is an unbiased estimator of o?. 


f) If the random variable z and the vector x are also related by E(z) = x7 B, 
then the linear function of the observed data y that has mean x7 B and has 
the smallest variance in the class of all such estimators is given by 


~ 


=x", 
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which has variance 
osia*x? (XIX) ax. 


g) The value Aur of B that minimizes S(B) subject to the k (< p) linear 
constraints HB =T is 


Bur = B+ (X7X)"'H[H(X7X)""H7]-“1(P — Hf) 
=B+ (X7X)"*HDy ren r, 


while ‘ 
RSS(H,T) = S(Byr) = RSS + eh rDurenr, 


where 


exp=I'-Hf and Dypr = [H(X7X)"!H7)-!. 


Implications: Part (a) explains why # is called the least squares estimator, 
while parts (b) and (c) explain statistically why we use B, namely that it is 
BLUE (best linear unbiased estimator). Part (d) gives further reason for using 
B, namely that it affords an orthogonal decomposition of the variability in 
y into that due to the deterministic and random parts. Part (e) provides a 
means of estimating 7? unbiasedly, while part (f) shows how to “predict” an 
unobserved value of the dependent variable when given a new set of values for 
the independent variables. Finally, part (g) gives expressions for the so-called 
restricted least squares estimator and its variance. 


The next two theorems allow us to perform tests of hypotheses or form 
confidence intervals if either the errors € are normally distributed (Theorem 
A.5.2) or certain conditions on y and X are satisfied that provide a central 
limit type theorem for regression (Theorem A.5.3). In Theorem A.5.2, we will 
phrase most of the results in terms of a linear function 6 = AB of B. This 
allows for more general results, but all of these results can be expressed about 
Bit A= 1s. 


Theorem A.5.2 | INFERENCES FoR NORMAL ERRORS 


Let A be a full rank (k x p) matrix of constants, and let 9 = AB, and 
6= AB. If€ ~ Nn(On,o7I,), then 


a) 6 is the maximum likelihood estimator of 6, 


6 ~ N,(0,07A(X7X)-!A7), 


392 MATHEMATICAL AND STATISTICAL TOPICS APP. A. 


and 6 is MVUE; that is, of all functions (not just linear functions) of y that 
are unbiased for 6, 8 has minimum variance. 


b) If we define the quadratic form 
Co(6) = (6 - 8)" V-*(6 —8), 


where V = A(X? X)-!AT = + Var(8), then we have 
o 


— or ~~ Xn-p 
Co(6) 
C2 a o2 ~ ko 


and C; and C2 are independent. This gives 


PLC kee G8) 
C,/(n — p) ke? 


Pad Fin-p, 


which in turn gives by Scheffé’s theorem (see Theorem A.4.3) that the proba- 
bility is 1 — a that simultaneously for all k-dimensional vectors h, 


h76 — S;(h) < h70 < h76 + S,(h) 


or 
h7@ — S>(h) < h76 < h76 + S2(h), 
where 
oy? 
2 = a,k 
alls hTV-1h 
ke? F 
2 a ajkn— 
Sah) bTV—1h 


c) If the matrix A is such that AB = B2 where B2 consists of k (< p) of 
the elements of B, and we let 8; represent the other p— k elements of B, then 


p — [RSS(B1) — RSS(B)I/k 
RSS(B)/(n — p) 


_ R(B2|B1)/k 


eo 


SEC A.5 LEAST SQUARES REGRESSION ANALYSIS 393 


while if Bz consists of a single element of B, G, say, then F = t?, where if we 
denote the rth diagonal element of (K7X)-1 by (X7X)=!, we have 


d) If HB =T, then 


_ [RSS(H,T) — RSS(B)]/k K 
RSS(B)/(n — p) " 


e) If Z is as in part (f) of Theorem A.5.1, then 
2 ~ N(x" B,o?x™ (X7X)=—1, x) 


2-—z~ N(0,07(1+x7(X7X)~1x)). 


Implications: Part (a) says that the least squares estimator @ is the best 
unbiased estimator of 8. Parts (b), (c), and (d) allow us to perform tests of hy- 
potheses and form confidence regions for regression parameters. For example, 
to test Ho : AB = A we would reject Ho if F > Fajkn—p, While a confidence 
region for @ is given by the set of vectors a satisfying C,(0)/ke? < Fen a2 
Results similar to the simultaneous confidence intervals in part (b) are used 
in Chapter 3 to find simultaneous confidence bands for autoregressive spectra 
(see Theorem 3.8.4). These Scheffé intervals allow the data to suggest interest- 
ing functions of the parameters to test. Part (e) provides confidence intervals 
for the mean E(z) = x7 of an unobserved value z of the dependent variable 
corresponding to a new value x of the independent variables: 


2 + tajon—p\/G2x? (XT X)-1x 


and prediction intervals for z: 


2+ ta/2n—p\/O2(1 + x? (XT X)-1x). 
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The Simple Form of the REG Command 


Most of the material in this section can be illustrated by using the general 
form of the REG command which we will introduce in Section A.5.5. For the 
linear model y = Xf + € where X is (n x p), if the arrays y and X contain y 
and X repectively, then the command 


REG(y,X,n,p, beta,RSS,t,e) 


will return B = (X?X)-!X?y, RSS =yTy — BPXTy,e=y—Xf, and 


Bj 


1} = een f=, ep 
RSS x 
Vane Mis 


in beta, RSS, e, and t respectively. See Section A.5.4 for an example of the 
use of this command. 


The Case of Nonnormal Errors 


If the errors in the general linear model are not normally distributed, we 
would hope that # might still have a normal distribution for large n. After 
all, B; is a linear function of y;,...,y, and thus of €,...,€,, and we know 
from elementary statistics that some linear functions of nonnormal random 
variables are approximately normally distributed. For example, the Central 
Limit Theorem is concerned with linear functions whose coefficients are all 
1/n. 

To obtain a Central Limit Theorem for regression analysis, we must make 
assumptions both about ¢ and about the linear function used to calculate B, 
that is, about X(X7X)-!'X?7. There are several sets of conditions in the 
literature, but we will follow Anderson (1971), p. 23. 


Theorem A.5.3 | CENTRAL LIMIT THEOREM FOR REGRESSION 


Let y = x7 B+, fort = 1,2,..., where the e’s are iid with mean zero 
and variance o?, and x1,X2,... are (p x 1) vectors of fixed constants. For 
n=1,2,..., let X, be the (n x p) matrix having tth row x7, and let aj; be 
the ith diagonal element of X7 X,, fori=1,...,p. Let 


R, = Dz!X?X,D;", 


where 


De Diag(a}!? ex ahl?): 


nl») “np }) 
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that is, the (i,j)th element of R,, is 
R,, —— (XP Xn)ij 
I (KEXn) ii (KFXn) 35 


that is, Rn is just a scaled form of J Oe 
Let B, and 62 be the ordinary least squares estimators of B and o? based 
on X, and yn = (y1,..., yn)”. If 


i) lim ay; = 00, fo Lee DN 
n-co 

ac “ z ; ” 

Balint oes Beal ira. 5D, 


N00 dn; 
and there exists a (p x p) nonsingular matrix R such that 
Ht} Hittin seo dtn dy = fey, 1,3 = 1, 2°": 
then 
a) Da(Bn — B) —+ Np(0p,0?R-}). 


b) 62 —, ¢? and 62 is asymptotically independent of By. 


Implications: These results essentially allow us to use all of the tests and 
confidence regions in Theorem A.5.2 except that now significance and confi- 
dence levels are valid only for large n. The conditions (i) and (ii) ensure that 
the weights applied to the e’s do not die out too quickly as n — oo. The matrix 
D,, plays the role that \/n does in the usual Central Limit Theorem; that is, 


Vn(X — p) “+ N(0,0?). 


A.5.3. The Case of Correlated Errors 


If y = XB +e, where Var(e) = o?V and V is a known (n x n) positive 
definite matrix, the results of the previous two theorems can still be applied 
by transforming the original model to one having uncorrelated errors. 

If we know V, we can calculate its modified Cholesky decomposition (see 
Section A.1.1) V = LDL? and its inverse square root V~1/?. If we define 
z=V1'2y we= V-1/2X, and n = V~1/%, then we have 


z= WB +n, 


where Var(n) = o7I,, and all of the results of the previous theorems can be 
applied to this new model. We summarize some of these results in the next 
theorem. 
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Theorem A.5.4 | GENERALIZED LEAST SQUARES 


Let y = XB +€ where € ~ (0,,07V). Then 
a) B = (X™V-!X)-!X?TV~-!y is BLUE for B, 
Var(8) = 07(X4V-4X)=1, 
and 
52 i y? Vly ¥ BIXxtTy 
n—p 


is an unbiased estimator of o?. 


b) Ife ~ N,(0n,07V), then 
B ~ N,(B,o7(X7V-*X)~*) 
Ca(B) = = (8 - 8)? XT V-*X(B— A) ~ x2, 
and C(f) is independent of 


(n — p)@? 2 
a ~ Xn—-p: 


If we don’t know V but we do know that its elements are a continuous 
function of a finite number r of parameters @, that is, V = V(@), and if 
consistent estimators @ of @ are available, then we have for example 


Co(B) = (8 - BT XTV-*()X(B — 8) > x2. 


This idea is used in Section 3.7.1 to analyze regression models having autore- 


gressive errors. 


A.5.4. Special Types of Regression Analyses 


In this subsection we consider orthogonal, stepwise, and recursive regres- 
sion. Each of these is useful in analyzing ordinary regression data but is also 


important because each has an analog in time series analysis. 


APP. 


SEC A.5 LEAST SQUARES REGRESSION ANALYSIS 397 
Orthogonal Regression 


A source of difficulty in regression analysis is that the elements of B are cor- 
related since Var(f) = o?(X7X)-1 which is not generally diagonal, and thus 
inferences made about one element are not independent of inferences made 
about other elements. One strategy for solving this problem is given by or- 
thogonalizing X. The Gram-Schmidt decomposition X = QR (Section A.1.2) 
can be used to do this because if we write 0 = RB, then we have y = Qé6+e 
and f satisfies 


X?X# = XTy 
R7Q7QRA = R7Q7y 
DR£ = Q’y, 
where D = Q7Q. Further, 
8 = (Q7Q)"'Q7y = D“1Q7y = Rf 


and 
Var(6) = 07(Q7Q)-! = o?D-!, 


which is diagonal, and thus the elements of 6 are uncorrelated (independent if 
the errors are normal). Thus one can make inferences separately about its ele- 
ments and then recover B = R16. In Section 3.4.4 we see that sample partial 
autocorrelations in time series analysis consist of the orthogonal parameters in 
a special kind of regression problem. 


Stepwise Regression 


Perhaps the most important use of the sweep operator (see Section A.1.3) 
is in stepwise regression. Given n observations on a dependent variable y and 
independent variables X,..., X,, the aim of stepwise regression is to determine 
a subset of the independent variables that adequately explains the variability 
in y. This is usually done by (see Jennrich (1977) for example) recursively 
adding and deleting independent variables from the model. 

The procedure begins with no variables in the model. At each step of the 
algorithm, a variable that is a contender to be added and one to be deleted is 
chosen (at the first step there is no deletion contender). Then the algorithm 
judges whether the deletion contender should in fact be deleted. If so, it is 
deleted and the algorithm goes to the next step. If not, then the algorithm 
checks whether the addition contender should be added. If so, it is added 
and the algorithm goes to the next step. This stepping stops when no more 
variables can be added or deleted. 
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The sweep algorithm is ideal for carrying out the steps described above. 
The algorithm begins by forming the matrix 


oe sh 
voy y 


The adding and deleting of variables are done recursively in A. Adding a 
variable that is not in the model is done by sweeping A on the diagonal cor- 
responding to its column in the X matrix. Deleting a variable that is in the 
model is done by sweeping on its diagonal element. 

Denote the matrix A after k steps of the algorithm by 


Bou 
A= 

Vid 
Then d is the residual sum of squares for the model chosen so far, while u and 
v are equal except that the elements corresponding to variables that are in the 
model have opposite signs. The elements of v have the correct sign. Thus at 
the kth step, suppose variables having indices ki,...,k, are in the model while 
those having k,41,...,kp are not. Let kp and k, denote the contenders for 


deletion and addition respectively. Then ka and kp are the values of j that 
maximize 


Uk: 
Maio shart 
Bx ;,k; 
and minimize 
lux, | c 
; Pe <p, 
Bx j,k; 


respectively. There are a variety of rules for addition and deletion but the one 
used in TIMESLAB is given by adding ky if 


2 P 2 
Xa,p—r Deri UR, 


GPS oie) as 
and deleting kp if 
2 
i X‘01,11%k | 
nh be GSH Cas Ne 


where Nees is the value of a coe random variable having area a to its right. 

Most implementations of stepwise regression also will not allow deletion of 
the most recently added variable, and will have a maximum number of steps 
to make sure that no infinite loop of adding and deleting can occur. 

Stepwise regression is very popular, but proving any sort of theoretical 
results such as p-values or confidence limits is very difficult. Traditionally 
it is regarded as an exploratory technique whose results must be confirmed 
statistically by obtaining more data. 
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The Stepwise Form of the REG Command 


If the arrays y and X contain the observations and design matrix for an 
(n x p) regression model, then the command 


REG(y,X,n,p,pval,kopt,k, beta, ind,RSS) 


will perform the stepwise regression procedure described above. The first six 
arguments are input. The argument pval corresponds to 1—a in the description 
above, while kopt is zero if the usual algorithm is desired, or a positive integer j 
if the user wants to “force into” the model the variables having indices contained 
in the array ind of length j. In either case, the output integer variable k is the 
number of variables in the chosen model (k=j if kopt>0), while ind and beta 
are output arrays of length k having the indices and coefficients in the chosen 
model. Finally, RSS is the residual sum of squares for the chosen model. 


An Example of the Simple and Stepwise Forms of REG 


To illustrate how the simple and stepwise forms of the REG commands can 
be used, consider the Hald data (Draper and Smith (1981), p. 296) given in 
Table A.1. Note that this table shows the data set exactly as it appears in 
the disk file called HALD.DAT. The first four columns are independent variables 
while the last is the dependent variable. 


Table A.1. The Hald Data 


Hald Data (X1,X2,X3,X4,Y) 


65 
a 226 6 60 78.5 
LE 2S0e 159 S32 74.3 
LiLp.e56. 8' 55205 104.3 
1 931 8 47 87.6 
Zz . 52 6 33 95.9 
11 3 =55 ce Prk ap loa 
os tL 17 Grier 
i $31722" 44 72.5 
2 54 1844:22 93.1 
2h eas, Awe 26 ae 115.9 
1. 4056523 e534 83.9 
11 66 eS gible pec) 
10 68 or rizewi09;4 


The following macro fits both the full model using the simple form and a 
subset model using the stepwise form. 
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MATHEMATICAL AND STATISTICAL TOPICS ATE. 


ae REG.MAC: macro to do regression analysis of data on the file 

sia fname. In that file, you need to put one observation 

36 per line, with the X’s first and then the y as the last 
ae entry in the line. The first line of the file should 

as be a title and the second line should be n*(mti) where 
ae n is the number of observations and m is the number 

$8 of regressors (not including an intercept). Note that 
re there should be no column of 1’s in the file. 
’ 


ag INPUT: fname, n, m, pval, kopt 

>> 

PAUSE 

;start 

PROMPTOFF 

READ (fname , X ,nn) j3nn should be n*(m+1) 

mp1i=m+1 ‘: 

X=TRAMS (X,mpi,n) ;this will get x in the right form 
one=LIWE(n,1,0) ;put intercept in 

X=<one,X> 

nnpi=nn+1 

nnpn=nnt+n 

y=EXTRACT(X ,nnpi ,nnpn) ;pull off y from end of X 
REG(y ,X,n,mpi,beta,rss,t,e) ;simple form 

LIST (beta) ;list coefficients 
LIST(t) ;list t-statistics 
LIST(e,n,7,7£8.3) ;list residuals, 7 per line 
’ 

;form and list estimate of covariance matrix: 
XTX=MMULT(X ,X,n,mp1) 

VAR=MINV (XTX ,mpi, ier) 

df=n-mp1i 

s2=rss/df 

VAR=s2*VAR 

label (VAR)=’Covariance Matrix of Estimates’ 
mpi2=mp1+*mp1i 

LISTC(VAR ,mp12 ,mp1 ,5£10.4) 

PAUSE 

;Stepwise form: 


REG(y ,X,n,mpi,pval,kopt ,k, betaa, ind,rssa) 


IF (k.eq.0,end) ;if no variables chosen, 
LIST(ind ,k) ;end macro. Otherwise, list 
LIST(betaa,k) ;output. 
XA=EXTRACT(X,n,1,1,ind,k) ;Pull off columns of chosen 
XTXA=MMULT (XA ,XA,n,k) ;variables. 
VARA=MINV(XTXA,k,ier) 

dfa=n-k 

82a=rssa/dfa 

VARA=s2a*VARA 


label (VARA)=’Covariance Matrix of Estimates’ 
k2=k*k 
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55 LIST(VARA,k2,k,5£10.4) 
56 ; 

57 ;end 

58 ; 


59 PROMPTO 
If we precede the invoking of the macro with the lines 


fname=’hald’ 
n=13 

m=4 

kopt=0 
pval=.95 


then the output generated by the stepwise form of the REG command is given 
in Table A.2. 


Note that the macro inserts a column of ones into the regression matrix 
and thus there is a total of five independent variables. These are labeled 1 
through 5 in the stepwise output. The stepwise regression begins with three 
lines that tell how many variables are in the full model and the maximum 
number of variables that can be in the final model. This maximum number 
is actually displayed as 0 if kopt is 0. Each step of the stepwise procedure 
is displayed between a pair of dashed lines. Included are the number of the 
variables and the value of the corresponding coefficients that are already in the 
model (there are no such variables during the first step), the value of RSS prior 
to beginning the step (this is labeled RVAR), and seven other variables. The 
variables labeled V ADD and V DEL are the numbers of the variables that are 
contenders for addition and deletion in this step. The variables labeled NPRED 
and NO CYC are the number of variables in the model prior to this step and the 
number of steps that have been made prior to this step respectively. Thus at 
the first step these are both 0. Finally, the variables labeled C ADD and Cc DEL 
are the criteria for addition and deletion; that is, they are C4 and Cp in our 
discussion of stepwise regression above. At a given step, the variable labeled v 
ADD can be added if the criterion is greater than 1, while the variable labeled 
V DEL can be deleted if the criterion is less than 1. Recall that the contender 
for deletion is checked first. Thus in the next to last step in Table A.2, the 
variable number 5 is in fact deleted. The stepwise procedure stopped in this 
example because it could not delete variable 2 because the criterion is 19.611 
_ (which is greater than 1), and it could not add variable 5 because it had just 
deleted it. Note that the model determined here is the same as that found by 
Draper and Smith (1981), p. 310. 
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Table A.2. Stepwise Regression Analysis of the Hald Data 


C ADD 


Cc ADD 


REGRESSION ESTIMATION STAGEWISE SUMMARY 
WUMBER OF VARIABLES IN FULL MODEL = 5 
MAXIMUM NUMBER OF COEFFICIENTS = 


LST 
0 


te) 


WO CYC 


WO CYC 


WO cyc 


HO cYc 


wo Cyc 


RVAR 
Oeeeeeeeee 


RVAR 
12713 .4470 


RVAR 
2 882.1382 


RVAR 
3 74.7965 


RVAR 
4 48.3989 


APP. A. 


Regressors Chosen 


1) 
a 


> | 
4 | 


VAR IW MODEL COEFF 
C DEL V ADD VDEL V 
1.493 1 to) 

VAR IW MODEL COEFF 

1 95 .43077 

C DEL V ADD V DEL V 
61.098 5 te) 

VAR IN MODEL COEFF 

1 117.57210 

5 ~ .73804 

C DEL V ADD V DEL V 
3.129 2 5 

VAR IW MODEL COEFF 

1 103.11730 

2 1.43839 

5 - .61397 

C DEL VADD V DEL V 
14.641 3 2 

VAR IW MODEL COEFF 

1 71 .89907 

2 1.45028 

3 -41306 

5 — .23933 

C DEL VADD \V DEL V 
-253 4 5 

VAR IH MODEL COEFF 

1 52 .60338 

2 1.46684 

3 -66210 

C DEL V ADD V DEL V 
19.611 5 2 

1.000000 2.000000 
Stepwise Regression Coefficients 
52 .603380 1.466842 
Covariance Matrix of Estimates 

4.3567 - .0405 -.0765 

-.0405 -0123 -.0011 

-.0765 -.0011 -0018 


Tol 


LST 
5 


3. 


WO cYc 


RVAR 
5 58.5657 


SEC A.5 LEAST SQUARES REGRESSION ANALYSIS 403 
Recursive Regression 


An important topic in statistics is recursive estimation. In regression anal- 
ysis this means given the results 8, RSS , and V = (X?X)-! of regressing the 
(m x 1) vector y on the (mx p) matrix X, we seek the corresponding quantities 
after adding or deleting data Y,, and X, of length n. Thus we seek 


By = (X™X + X7X,)"1(XTy + XTy,) 
RSSy = (y7y tyty1) — BY (X7y + X7y;) 


Vn = (X7X + X7X))“1, 


where the + is + for adding observations and — for deleting. From the matrix 
inversion lemma (Theorem A.1.4) we can obtain recursive formulas for these 
quantities. 


Theorem A.5.5 | RECURSIVE REGRESSION FORMULAS 


The updated quantities By , RSSyn, and Vy defined above are given by: 
By = 6+ AX7 (I, + H;)~'e; 


= B+(X7X4+xX?X,)-1xTe, 
RSSn = RSS ae ef (In ae H,)~ te; 
= RSS + ef (I, = X1(X7X + X7X,)"'X7 Je, 
Vn =AFAX{ (I, + Hi)“ XA 


il P0965 ip pat 
where 


H,=Xi;AXi, A=(X7X)-?, ec, =yi—X,f. 
Proof: The basis of this theorem is the matrix inversion lemma and the facts 
(easily verified by multiplication): 
(I, + H,)~*=1, + (In + Hi)“' Hi = 1, + Hy (I,, + Hi)-?. 
Thus 
Bn = (X™X + X?X,)-1(X7y + X7Ty}) 


= [A = AX? (I, + Hi)~'X,A](X7y + X7y,) 


= B+ AXT (I, + Hi)“1y1 + AXTy; — AX7(I, + Hi) Hyy;, 
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where y; = X; AX7y = X;8, which gives 
By -B =+AXT7[L, + (In + Hi) ~My 
+ AXT (I, + Hi) “yr 
= tAXf (I, + Hi)“ ‘(yi — 91), 
which gives the first expression for By. To show the second, note that 
(X7X + X?X,)7'XTe. = [A = AX? (I, + Hi)7?X, A]X7 ee, 
= AXT[I, # (In + Hi) Hi Jer 
= AX7 (I, + H;)~1e,. 
To find RSSyn we have that +Hiy, = (I, + Hi)yi — yi, and so 
RSSy = (y7y tyty1) — (87 te? (I, + Hi)“*Xi A](XTy + XTy;) 
= RSS tylyi $B XT y, F ef (I, + Hi)~1 (Ki 8 + My) 
= RSS ty{yi $B’ XTy1 F ef (In + Mi) ty 
¥ ef (In + Hi)" [(In + Hi)yi — yi] 
= RSS + (yf — B’XT — ef )y: ¥ ef (In + Hi)(yT — y1) 
= RSS +e7 (I, + Hi)~'e, 


since ef = y? — BT X7. To obtain the second expression for RSS N, note that 


ef (In # X1(X7X + X7X,)-'X7]ey 
= ef {I, = Xi[A = AX? (I, + Hi)~1X, A] X? Jey 
= ef {I, Xi AX? +X AX? (1, + Hi) Xi AX7 Je, 
= ef {In ¥ Hill, F (In + Hi)“ HiJ}er 
= ef {1, + Hi (I, + Hi)“ }ex 
=e?’ (I, + H,)~1e1. 


Finally, the first expression for Vy follows from the matrix inversion lemma, 
while the second is by definition. 
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Note that in each pair of equations for the updated quantities, the first 
equation is appropriate when n < p while the second is for n > p. The sweep 
operator can be used in either case. For n < p, we have 


ee x) fo 


SWEEP(1,...,p +n) 
FXy in D E 


ka =e x7(E ae H,)-1X,A —AX? (I, 2 Ten 


+(In + H,)-!X, A (In + Hi)-? 
and thus 
By = BF Cey 
RSSn = RSS +e? Ee, 
Vn =B. 
For n > p, 


SWEEP(I,...,p) 


X?X 35 x?X, | | U " 


te? X, ele; = 


Ww z 
(X?X + X7X,)"} (X7X + X7X,)-!x7e, 
a pee (X7X4X7X,)-! ef fl, ¥Xi(X7X + X?X,)-X7]e, | 
and thus 
Bu =B+tv 
RSSy = RSS +z 


Vn = U. 


A.5.5. The General Form of the REG Command 
The general form of the REG command is: 
REG(y,X,N,p,beta,Q,RSS,ni,t,e,lrsio[,V]). 
The input integer N can be either positive or negative. Its sign is used as a 
signal to REG, while its absolute value is the number of elements that the array 


y has and the number of rows that the matrix X has. Thus let n=|N| and i=1 
if N is positive and i=0 if N is negative. 
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The arguments y (n x 1), X (n x p), N (integer), and p (integer) are input; 
the arguments t (p x 1), e (n x 1), and 1rsio (real variable) are output; while 
beta (p x 1), Q (p x p), RSS (real variable), and ni (integer variable) are both 
input and output. If n < p, then the (n x n) matrix V is both input and output, 


while V is ignored if n > p. 


For the purposes of this discussion, we will attach a subscript I or O 
(denoting input and output) to any argument that is used for both input and 
output. There are two cases that REG handles, namely when the number of 
observations is greater than the number of parameters and when the number 


of observations is less than or equal to the number of parameters. 


Number of Observations > Number of Parameters (n > p) 


In this case, REG uses the following formulas: 


Bo =6rt+iv 

Qr+iX?X, ni, £0 
Qo= 

ix xX, nip 0 


RSS; +is, ni, #0 
RSSo = 
is, ni7—0 


nig = ni;y+in, 


while 


e=y-XBo 


lrsio = log(RSS; + ie? e)) — log(RSSo), 


and U, u, v, and s are calculated by 


with e; = y — Xf_. 
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Number of Observations < Number of Parameters (n < p) 
In this case, we have: 
Bo = Br — iBre; 
Qo = Az 
RSSo = RSS; + ie? Doe; 
nig =ni;+in 
Vo =D), 


while 


1rsio = log(RSSo) — log(RSSz), 


where e; = y — Xf, and 


be Be i | 
SWEEP(I,...,p) = 
iY 


Ai B, A» By 
SWEEP(p+1,...,p+n) ze 
C, D, C2 Dp» 
Ay Bo 


SWEEP(1,...,p) | 


APPENDIX B 


User’s Guide to TIMESLAB 


Throughout this book, the TIMESLAB program is used in examples and 
exercises. In this appendix and the next we provide the actual manual for the 
use of the program. This appendix is called the user’s guide as it discusses 
installing the program, getting started in the process of using it, and its ca- 
pabilities in general. Once the program has been installed, we strongly urge 
the reader to work through the “guided tour through TIMESLAB?” given in 
Section B.3. In Appendix C, we provide detailed descriptions of each of the 
TIMESLAB commands. 


TIMESLAB is called a laboratory because its aim is not so much to provide 
“canned” time series analysis programs (although it can be used that way), as 
it is to provide a setting in which various building blocks of time series analysis 
can be combined to study a series or method in depth. ZTIMESLAB is an 
integrated system in that it has its own editor, several high resolution graphics 
commands, the ability to read from and write to files, and the ability to issue 
DOS commands without exiting from the program. 


Single commands exist to read or save data, to calculate a wide variety 
of quantities in both the time and frequency domain for either univariate or 
bivariate time series. Text and graphics output can be merged on a printer, 
and graphics screens can be saved onto files for later display. There is also the 
ability to process files called macro files that contain a series of TIMESLAB 
commands. This ability means that TIMESLAB can be used as a time series 
analysis programming language. 


B.1. Hardware and Software Requirements 
To use TIMESLAB, one needs: 


1. An IBM PC, PC/XT, PC/AT or any personal computer that is compatible 
with the IBM family of PC’s. 
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2. 640K RAM. 


3. The IBM Color Graphics Adapter (CGA) or Enhanced Graphics Adapter 
(EGA) or any adapter compatible with these. 


4. A monitor that can perform pixel graphics and is compatible with the 
graphics adapter that is installed in the PC. 


5. Either two floppy disk drives or a hard disk. 


6. An 8087 (or 80287 if one has a PC/AT or AT compatible) numeric copro- 


cessor. 


7. For printed output, any nine pin dot matrix printer that recognizes the 
printer commands of the IBM Graphics Printer, any 24 pin dot matrix 
printer compatible with a Toshiba 321 Printer, or any laser printer com- 
patible with the Hewlett-Packard Laserjet or Laserjet Plus Printer. 


8. Version 2.0 or later of DOS. 


9. When the computer is started by turning it on (this is referred to as a 
“cold boot”) or restarted by issuing the Ctrl-Alt-Del “warm boot” from the 
keyboard, two files must be on the root directory of the “booting” device 
(i.e., the hard disk if there is one or a floppy which has been formatted 
with the /S option in the DOS FORMAT command): (1) the file called 
ANSLSYS which comes with DOS, and (2) a file called CONFIG.SYS 
containing the DOS command 


DEVICE=ANSI.SYS. 


The presence of these files makes it possible for TIMESLAB to use the colors 
(or shades of green or amber on a monochrome monitor) in the text mode. 
No other software is required as TIMESLAB is designed as an integrated sys- 
tem; that is, it has its own full screen editor, its own graphics, and its own 
communications to the printer. Any editor or word processor can be invoked 
from within TIMESLAB by issuing the DOS command first. This command 
establishes a modified DOS prompt from which any command issuable from 
DOS can be issued without affecting what is being done in TIMESLAB. 


B.2. Installing and Starting TIMESLAB 
Everything that is needed to use TIMESLAB is contained on the two 


diskettes accompanying this book. TIMESLAB is not copy protected in any 
way. 
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Diskette 1 


The only file on this diskette is TIMESLAB.EXE which is the executable file 
containing TIMESLAB. 


Diskette 2 
This diskette contains the following files: 


1. TSLABHLP: This file contains the text of the help messages in TIMES- 
LAB. Note that TSLABHLP is not printable. 


2. HELP: This is a printable version of the help file. 


3. TSLABWI1: Any file having this name can be formed by the user and its 
contents will be displayed on what is called a “second welcoming screen.” 
See below for more details on the purpose of this file. 


4. All of the data sets referred to in this book. These are files having exten- 
sion .DAT. The first part of their name indicates which of the data sets 
discussed in the book that they are. See Appendix D for a listing of each 
of the data sets. 


5. All of the macros discussed in the book. These are the files having exten- 
sion .MAC. See Appendix E for an index to the macros. 


6. A few files having extension SCN. These files contain graphics screen 
images. They can be “re-screened” using the command RESCREEN. 


7. A file called EXAMPLE.NTE. This file is an example of what is called a 
notes file and is used with the NOTES command. 


The TSLABW1 File 


When TIMESLAB is started, it displays a welcoming screen and waits 
for the user to strike a key to continue. If there is a file called TSLABW1 
on the current subdirectory of the default disk drive, then a second welcom- 
ing screen is displayed containing the contents of TSLABW1. If not, then the 
second welcoming screen is not displayed. Users can form their own file called 
TSLABW1 containing, for example, messages to someone else using TIMES- 
LAB. An instructor for a time series course could create a file containing notes 
to the students taking the course. 
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Installing TIMESLAB 


The first thing to be done is to make backup copies of the distribution 
diskettes. See the beginning of any DOS manual for a description of how to do 
this. Once these copies are made, the method in which TIMESLAB should be 
installed depends on whether the user has a hard disk or not. 

There are three basic ways to use TIMESLAB: 


1. On a two-floppy system. 
2. On an unshared hard disk system. 


3. On a shared hard disk system. 


Two-Floppy System 


In this situation two diskettes should be formed, one called the A floppy 
which contains a copy of COMMAND.COM (so that the DOS command will 
work) as well as TSLABW1, TSLABHLP, and any .MAC, .SCN, .NTE, and 
.DAT files to be used, and the other floppy called the B floppy which contains 
only TIMESLAB.EXE. Once this is done, with the DOS prompt being A>, the 
command 


B: TIMESLAB 


will start TIMESLAB. Setting things up this way assures that there is sufficient 
room on the default floppy (i.e., floppy drive A) to receive data, macro, and 
screen files. Once TIMESLAB has been started, the program diskette in drive 
B can be removed and both drives can be used to contain data sets, macros, 
and screen images. 


Unshared Hard Disk System 


In this situation, a subdirectory should be formed containing everything 
on the distribution diskettes. Then TIMESLAB can be invoked from a prompt 
designating the hard disk by just saying its name. 


Shared Hard Disk System 


In some cases readers will be using a copy of TIMESLAB residing on a 
hard disk that someone else put there. In this situation, a floppy can be used 
to receive any files such as macros, data files, or screens produced during a 
TIMESLAB session. Thus a floppy should be created containing anything that 
is needed (it doesn’t need TIMESLAB.EXE or COMMAND.COM), the DOS 
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prompt should be set to A>, and then the command (assuming that TIMES- 
LAB.EXE is on hard disk drive C) 


C: TIMESLAB 
will start TIMESLAB. 
Getting Colors Other Than Black and White 


In order to get colors other than black and white in the text mode of 
TIMESLAB, the PC being used must have been booted with a file called 
CONFIG.SYS residing on the root directory of the booting disk drive and 
CONFIG.SYS must contain the line 


DEVICE=ANSI.SYS. 


Further, the file called ANSI.SYS that comes with versions 2.0 and later of 
DOS must also be on the root directory of the booting disk. If this is not 
the case, no colors will be produced in the text mode and strange characters 
will appear on the screen whenever TIMESLAB switches from graphics to text 
mode. Note that these characters will appear on the printer sometimes in any 
event. 


B.3. A Guided Tour Through TIMESLAB 


To get an idea of how TIMESLAB works, we suggest that you work 
through the following tour of the program. This tour only considers a small 
fraction of the capabilities of the program but should be useful in getting 
started. For the tour to be successful, the following must be true: 


1. You are using an IBM-compatible PC or PC/XT or PC/AT with 640K 
RAM that has a color graphics adapter or enhanced graphics adapter and 
either a color monitor or monochrome monitor that can perform pixel 
graphics. 


2. You have booted your PC with a file called CONFIG.SYS (that you cre- 
ated) and a file called ANSI.SYS (that comes with DOS) on the root 
directory of the disk that you booted from, and included in the file CON- 
FIG.SYS is the line DEVICE=ANSI.SYS. 


3. You are logged on to the subdirectory that contains all of the files on 
distribution diskette number 2 and can access the file TIMESLAB.EXE. 
If you are using floppys this usually means that diskette 2 is in A and 
diskette 1 is in B. 
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The Two Welcoming Screens 


If these steps have been performed, then it is time to start TIMESLAB. 
Start the program according to the instructions in Section B.2. The first thing 
that you will see is a screen welcoming you to the program. On a color monitor 
this screen will have a blue background with a primarily yellow foreground. On 
a monochrome monitor, these colors will be represented by different shades of 
the color used by the monitor. If you did not get these colors, then you did 
not boot your PC with the files called ANSI.SYS and CONFIG.SYS set up 
properly as described above. If this is the case, strike the enter key twice, type 
the command QUIT, and then strike the enter key again. This returns you to 
DOS and you can review Section B.2 on how to set up your files properly. If 
you did get the colors and you don’t like them, you can change them later by 
using the TEXTCOLOR command. 

On the bottom of the screen is a message directing you to strike any 
key to continue. Do this. The next screen that you see is an optional second 
welcoming screen. At the top of the screen is a heading telling you that you are 
looking at a second welcoming screen, and at the bottom is a message saying 
to strike any key to continue. The contents of this screen are contained in the 
file TSLABW1 that came on distribution diskette 2. If you did not get this 
screen, then you are not logged onto a subdirectory containing the files that 
came on diskette 2, and this tour through TIMESLAB will not work properly. 
If this is the case, then you are probably looking at a screen with a question 
mark at the bottom. Type the command QUIT and strike the enter key. This 
should return you to DOS and you can return to Section B.2 to set up your 
files correctly. If you did get the second welcoming screen, strike any key and 
the second welcoming screen will disappear and a list of commands and the 
TIMESLAB prompt will appear on the screen. The PC will beep to indicate 
that it is now ready for you to enter commands. 


The List of Commands and the TIMESLAB Prompt 


You should now be looking at a screen that has a list of the 150 or so 
commands that are available. At the bottom of this list are two lines that tell 
you how to get information about the commands and then finally a question 
mark with the cursor next to it. This question mark is called the TIMESLAB 
prompt and is analogous to the DOS prompt (such as A>). You can get this 
same screen any time the prompt is on the screen (in the text mode, see below) 
by entering the command HELP. Note that entering a command means to type 
the command and then strike the enter key (also called a carriage return). 
In the rest of this tour through TIMESLAB, things that you will type will 
be represented by typewriter font (like this). We will always represent the 
command names by typing them in uppercase, although you can enter them in 
upper or lowercase. You can leave TIMESLAB and return to DOS by entering 
the command QUIT at any time. 
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The Text and Graphics Modes 


The monitor that is connected to an IBM PC and operated by a color 
graphics adapter can operate in two “high resolution” modes; (1) a text mode 
which consists of 25 rows of 80 character locations, and (2) a graphics mode 
which consists of 200 rows of 640 pixel locations. A character is actually made 
up of an 8 x 8 pixel box, but the individual pixels are only addressable in 
the graphics mode. On a PC equipped with the Enhanced Graphics Adapter 
(EGA), there are more pixels on the screen than the 640 x 200 for the CGA 
but TIMESLAB does not make use of this extra resolution. 

The two modes are different in the following ways: 


1. High resolution graphics can only be formed in the graphics mode. If one 
of the graphics commands is issued from the text mode, then TIMESLAB 
(1) switches to the graphics mode (which erases whatever is on the screen), 
(2) produces the plot, and (3) waits for the user to signal that he or she is 
finished viewing the plot, at which point it switches back to the text mode 
(which erases the plot). 


2. In the text mode, the TIMESLAB prompt moves down the screen until it 
reaches the bottom at which point the entire screen is scrolled up. In the 
graphics mode, the prompt is reissued at the top of the screen so that it 
does not overwrite any graphics that may be on the screen. 


Thus unless one wants to be able to have more than one graph on the 
screen at the same time, it is best to leave TIMESLAB in the text mode. 

When TIMESLAB is started, it is in the text mode. To switch from the 
text mode to the graphics mode, one issues the command PLOTON. Do this 
now; that is, type PLOTON and then strike the enter key. If you have a CGA, 
the screen should be black with a brown prompt. If you have an EGA, the 
screen should be brown with a green prompt. In the graphics mode you can 
get only two colors on the screen at once, one of which is called the background 
and another called the foreground. If you have a CGA, enter the command 
COLOR(2), and if you have an EGA, enter the commands COLOR(256) and 
COLOR(0). Now the prompt should be green for either adapter with a black 
background. You can experiment later with setting graphics colors with the 
COLOR command. Now enter the commands 


x=WN(0, 100) 
PLOT (x) 


You should have a graph of a white noise time series on the screen and the 
prompt still at the top of the screen. In the graphics mode, it is always issued 
at the top of the screen. Otherwise it would interfere with any graphs that are 
on the screen. 


SEC B.3 GUIDED TOUR 415 


To switch back to the text mode, the PLOTOFF command is used. Do 
this now. Now the screen should be blue with a white foreground. These 
colors can be changed by using the TEXTCOLOR command. Enter the command 
TEXTCOLOR(7,1). This makes the backgound white and the foreground blue. 
To get back to the original text colors, enter the command TEXTCOLOR(1,7). 


Getting Help 


The TIMESLAB help system can be accessed at any time that the program 
is in the text mode and prompting you for a command. Enter HELP now. Now 
enter HELP(wn). You should have a short description of the WN command on the 
screen. Each command is more fully described in Appendix C and somewhere 
else in the book where the method used in the command is described (see 
Appendix E for an index of commands). If you got a message saying that 
TIMESLAB couldn’t find the file TSLABHLP, then you haven’t set up your 
files properly and you should leave TIMESLAB and do so. 

Now enter HELP again. This time notice the message at the bottom of 
the screen saying that you can get a list of general help areas by entering the 
command HELP(areas). Do this now. Notice the list of areas. Now enter the 
command HELP(matrices). This will give a list of the commands that can be 
used to study matrices in TIMESLAB. Now enter HELP(estimation). This 
gives a list of commands for estimation. 


Data Types 


Now we will define some variables. Enter the following commands one at 
a time. Make sure they are entered exactly as given. In particular, make sure 
that the decimal point is included in third and fourth commands. If you make 
a mistake in entering a command, TIMESLAB will alert you to the fact and 
you should be able to just enter it again. 


k=10 
m=5 

c=1, 

=5. 
twopi=2.*pi 
fname=’air.dat’ 
READ(air,x,n) 
fni=’ wn#k#’ 
t=<1,2,3,4,5> 
LABEL(t)=’The Numbers 1 Through 5’ 
y=WH (12345 ,100) 
w=C0S(100,1,10) 
z=ytw 

INFO 


You should see the following on your screen: 
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Number of arrays and Free Elements: 5 9551 
WAME LENGTH LABEL 


x 144 International Airline Passengers 
t 5 The Numbers 1 Through 5 

y 100 normal white noise series 

Ww 100 cosine curve 

Zz 100 sum of two arrays 


k = 10 m = 5 
n = 144 

c = 1.00000000 d a 5 .00000000 
twopi = 6 . 28318500 

fname = air.dat fni = wnid 


You have defined the integers k and m, the real scalars c, d, and twopi, 


the character variables fname and fn1, and the four arrays y, z, w, and t. The 
READ command resulted in the integer n and the array x being defined as well. 
The INFO command will tell you all of the variables that you have defined at 
any time. The basic rules are as follows (much more detail is given in Section 
B.4): 


tl. 


Variable names can have as many as 15 characters, you can have as many 
as 60 of each of the four types, and the total number of elements for all 
of your arrays must be less than or equal to 10,000 (INFO tells you how 
many “free elements” you have left). 


. Each array has a 40-character “label” associated with it. This label is used 


at the top of lists and plots of the array. When you read a data set from 
a file (via the READ command as you entered above for the International 
Airline Data on the file AIR.DAT), the label that is in the file (see Section 
B.4.8) is the one that TIMESLAB uses. When an array is created by a 
command (such as the command y=WN(12345,100) above), TIMESLAB 
creates its own (often not very informative) label for it (see the labels for 
the arrays y, z, and w). This label can be changed by the LABEL command 
(see the one that you entered above for t and the result in the output of 
INFO). 


. TIMESLAB could tell that you wanted k and m to be integers and c andd 


to be reals because of the absence or presence of a decimal point. On the 
other hand, when TIMESLAB is expecting a real as an input argument, 
and you want to enter a real that is a whole number, you need not enter 
the decimal point (the first argument in the WN command was a real). 
There were two other things to notice in the example above: (1) the value 
of m is always available in the variable pi (therefore don’t define a variable 
called pi), and (2) in the line defining fn1, the value of k is inserted into 
the character variable. This is often a useful capability. 
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Listing Variables 


To list the values of arrays or other types of variables, we have the LIST 
command. Enter the command LIST(k,fni,c). You should see the values of 
these variables. Now enter LIST(x,50). This will display the first 50 elements 
of the airline data (as well as its label). Now enter LIST(y,100,10,10£6.2). 
This will list the normal white noise array with 10 elements per line in the 
Fortran format 10f6.2. Note that you can’t list both scalars and arrays in one 
command. 


Graphics 


There are several graphics commands in TIMESLAB. The simplest is the 
PLOT command, which does a simple Y versus X plot. There are several forms 
of this command (see Appendix C). We illustrate the PLOT command by plot- 
ting the airline data. Enter the command PLOT(x,n). You should see a graph 
of the data (on the vertical axis) versus the index 1 through 144 on the hori- 
zontal axis. At the bottom of the screen is what is called the graphics menu, 
giving you a choice of several keys that you can strike. (Do not strike any of 
these keys now.) The graphics menu is described in detail in Section B.5.3. 
Note the following: 


1. The label for this array is at the top of the screen. 
2. There are ten tic marks on each axis. 


3. The ends of the axes correspond to the largest and smallest values of the 
arrays being plotted. 


4. The numerical labels on the tic marks are “ugly,” particularly those on the 
horizontal axis which we would like to be whole numbers. We will correct 
this in a minute. 


5. There are no descriptive labels below the horizontal axis or to the left of 
the vertical axis. Such labels can be added to a plot either using the FIND 
utility (see Section B.5.5) or using the LABEL command. 


We will next redraw the plot that is on your screen. Notice that the values 
on the vertical axis range from 104 to 622, while those on the horizontal axis 
range from 1 to 144. There is another form of the PLOT command where you 
can specify the values that you want at the ends of the axes. Since there are 
ten tic marks, if we put 100 at the bottom of the vertical axis and 700 at the 
top, then the plot will fit and the tic mark labels will be 100, 160, 220, and so 
on. Similarly, if we specify that the ends of the horizontal axis are to be 0 and 
150, then the labels will be 0, 15, 30, and so on (except that only every other 
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tic mark gets labeled). Now strike any key. Enter the command 
PLOT(x,n,0,150,100, 700) 


and see how much better the tic mark labels are. This also makes it so that the 
plot moves away from the axes slightly. Note that the size of plots, as well as the 
number of tic marks and the number of digits in the numerical labels placed at 
the tic marks, can be specified using the PLOTSIZE command. Certain standard 
sizes and locations of plots have been defined in TIMESLAB. To illustrate this, 
enter the commands PLOTSIZE(4) and PLOT(x,n,0,150,100,700), and note 
that the graph is now in the lower right quarter of the screen. Now enter 
the command PLOTSIZE(0), strike the enter key again, and go on to the next 
section. 


Using Macros 


The ability of TIMESLAB to read commands from a file is what makes 
it so powerful. Such a file is called a macro. To illustrate this, we will use 
the WNTEST macro (it resides in the file WNTEST.MAC that came on the 
second distribution diskette and is discussed in Example 1.5) to determine if 
there is significant autocorrelation in the airline data. First, find the log of 
the data by entering the command x=LOGE(x,n) (the reason for taking the log 
is discussed in Section 1.5). Issue the command MACRO(wntest). You should 
see several lines on the screen starting with two semicolons and containing a 
description of what the macro does and what you have to define in order to use 
it. This is called the “prolog” of the macro and all of the macros that come with 
TIMESLAB have such a prolog. The last line should say PAUSE... STRIKE 
F9 TO BREAK, ANYTHING ELSE TO GO ON. The WNTEST macro expects 
that you have defined three variables: (1) an array x containing data, (2) an 
integer n containing the number of elements of the array x to analyze, and (3) 
an integer m containing the number of correlations to use (don’t worry about 
what this means for now). We defined x and n when we read the airline data, 
and we defined m to be 5, which is too small a value to be used in this context. 
Thus strike the F9 function key to “break out” of the macro. The TIMESLAB 
prompt now appears. Enter the command m=30 (no decimal point since this 
is an integer). Now restart the macro (right where it left off) by entering 
the command MACRO. Now TIMESLAB starts executing one command after 
another from the WNTEST.MAC file (the commands should go by quickly). 
Finally, the macro will pause with a graph on the screen with two horizontal 
lines and another curve above them. If there were no correlation in the data, 
this curve should be between the horizontal lines. Now strike any key (not F1, 
F3, F5, or F9 please), and the macro will resume execution, and finally you 
should see another graphics screen with three diagonal lines and another curve 
above them. Again, if there were no correlation in the data, this curve should 
be between the diagonal lines. Now strike any key and note that the macro 
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has ended because the TIMESLAB prompt has reappeared. 


Another Macro 


Another interesting macro is the one called COS.MAC (see Example 1.4). 
This macro forms four cosine curves of various periods and amplitudes, plots 
all four on the screen at once, then adds them together and plots the resulting 
data and periodogram on the screen at the same time. To see this, enter the 
command MACRO(cos). You will see the prolog and the message to strike any 
key to continue. Do this. Now you will see the lines of the macro at the top of a 
graphics screen as they are being executed, and finally the four cosine curves are 
plotted on the screen together with the graphics menu. Strike any key (except 
F1, F3, F5, or F9), and you will see the data and the periodogram plotted 
on the screen. Note that the periodogram has peaks at the four frequencies 
corresponding to the periods of the four component cosine curves. Now strike 
any key and the TIMESLAB prompt should reappear in the text mode. 


The FIND Utility 


An important feature of the graphics in TIMESLAB is the presence of 
an interactive locator/editor from any graphics screen. To illustrate this, enter 
the commands PLOTSIZE(0) (which resets the size parameters) and PLOT(x,n). 
You now should have a plot of the airline data that fills much of the screen. 
Part of the graphics menu is the option of striking F1 to enter the FIND utility. 
Do this. A box (called a “locator”) should appear in the upper right corner 
as well as the values of x and y where the locator is. Note that at the bottom 
of the screen is the FIND menu. This menu tells you that you can leave the 
FIND mode (don’t do this now) by striking Alt-F10 (i.e., hold down the Alt 
key while striking the F10 key, and then release the Alt key), or you can get 
directions on how to use FIND by striking Alt-F1 (don’t do this either). Now 
strike the key labeled 2 on the numeric keypad (not the one along the top of 
the keyboard). The locator should move down a little and the values of x and 
y in the upper right-hand corner of the screen should adjust to the move. If 
instead of the locator moving, a 2 appeared on the screen, then strike the Num 
Lock key and try it again. Note that you can move the locator in any direction 
by using the corresponding key on the numeric keypad (e.g., striking 7 moves 
it up and left, striking 1 moves it down and left, and so on). Now strike Ctrl-4; 
that is, hold down the Ctrl key while striking the 4 on the numeric keypad. 
The locator should have moved all the way to the vertical axis. Note that the 
numbers must be entered from the numeric keypad. 

Now type your name on the screen. Note that any of the 256 characters 
can be placed anywhere on the screen. If a character has a key, just strike its 
key. If not, then you must enter its ASCII code by holding down the Alt key, 
typing the ASCII code for the character, and then releasing the Alt key. When 
the Alt key is released, the character will appear. To illustrate this, we will 
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place an @ on the screen. We need to find the ASCII code for an a. Strike 
Alt-F1 and notice that you now have a help screen. Note that there are a 
variety of other things mentioned on this help screen, including directions for 
drawing lines or rectangles. At the bottom of the screen is a message saying to 
strike F1 to get a list of ASCII codes. Do this. Look up a in the table that is 
now on the screen. It should be character number 224. Now there is a message 
at the bottom of the screen saying to strike any key to return to FIND. Do 
this. Now strike Alt-224 (i.e., hold down the Alt key, type 224 on the numeric 
keypad, and then release the Alt key) and see how the a appears on the screen. 
There are a number of other features of FIND (see Section B.5.5), but for now 
we leave it. Strike Alt-F10 and note that you are back to the graphics menu. 
Strike any key and we’re back at the TIMESLAB prompt. 


Retrieving Graphics Screens 


To illustrate how graphics screens that have been previously constructed 
and saved can be instantly redisplayed, enter the command 


RESCREEN(smooth.scn) 


This should place a copy of Figure 1.10 of the book on your screen. Now strike 
a key to get rid of this plot and then enter the command 


RESCREEN(wind1.scn,wind2.scn,wind3.scn, 1) 


and see what you get. Now strike the enter key to return to the TIMESLAB 
prompt and go on to the next section. 


The DOS Mode 


It is possible to issue DOS commands while within TIMESLAB. To illus- 
trate this, enter the command DOS. A prompt will appear which is a combina- 
tion of the TIMESLAB prompt and the DOS prompt. There is also a message 
telling you to enter the command EXIT (in either upper or lowercase) in order 
to get back to the TIMESLAB prompt. Now issue the command 


DIR *.DAT 


and you should get a list of all the data files that were on the distribution 
diskette. Now enter EXIT and the ordinary TIMESLAB prompt should reap- 
pear. 
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Two Fun Macros 


To finish this tour, we will use two other macros: (1) CIRCLE, and (2) 
ODE (see Examples B.5 and B.4 respectively). Enter MACRO(circle, start) 
(the second argument tells TIMESLAB to begin execution of the macro at 
the line ;start in the CIRCLE macro—this skips the prolog). You will get a 
solid circle on the screen. Now strike the enter key and enter the command 
MACRO(ode,start). What do you get? 


Exiting from TIMESLAB 


To leave TIMESLAB and return to DOS, enter the command QUIT. Note 
that none of the variables that you have defined will be available at your next 
session unless you save them with the SAVE command. 


The Next Step in Learning TIMESLAB 


At this point, you should have a rough idea of some of the basic features 
of TIMESLAB. The next step is to learn more commands, to learn how to 
use the TIMESLAB editor (see Section B.6) so that you can form your own 
macros (see Section B.7 for details on macros) and data sets (see Section B.4.8 
for details about data sets), and to study the rest of this appendix for more 
details about the general features of the program. 


B.4. Overview of TIMESLAB Capabilities 


In this section we describe the general features of TIMESLAB. We begin 
by describing the five types of data recognized by TIMESLAB and the five 
types of commands that it has. 


B.4.1. Data Types 
There are five types of data recognized by TIMESLAB. 
Immediate Values 


These are integers such as 2, 7, and 777, real numbers like 3.88, -75.90, and 
3.1415926, and character values like ’character’, ’obs.dat’, ’hello mom’. 
Note that character immediate values are just strings of characters enclosed 
in apostrophes. When used as arguments in a command, immediate charac- 
ter values need not have the apostrophes. Integers are between —32,767 and 
32,767, reals between around 10%” and 10°’, while characters can have at most 
15 characters. 
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Character Variables 


These are variables whose names have at most 15 characters and whose 
“value” is a string of characters containing at most 15 characters. Character 
variables are formed either as output of a command or by a simple assignment 
statement of the form 


name=’string’ 


which is a type 2 command and is described below. Note that the command 
above sets a character variable equal to an immediate character value. 


Integer Variables 


These are variables whose names have at most 15 characters and whose 
value can be any valid two-byte integer (—32,767 to 32,767). For example, 
in the command n=3, n is an integer variable while 3 is an immediate integer 
value. 


Real Variables 


These are variables whose names have at most 15 characters and whose 
value can be any valid four-byte real value (approximately 10~%% to 10%8). 
There are two real scalar variable names that are always available, namely pi 
and exp which have the values 7 and e. Note that one can use an element of 
a real array as a real scalar by using the notation x[i]. For example one can 
say x[i]=y[10]+z[j] where x,y, and z are real arrays. 


Real Arrays 


These are variables whose names can have at most 15 characters and which 
contain sets of real numbers. Real arrays are the basic objects manipulated by 
TIMESLAB,; a time series data set is stored in an array, values of a correlogram 
or spectral density are arrays, and matrices are stored in arrays. Arrays can 
be formed either by a command of the form 


A=<1,2,3,4,5,6> 


or as output of a command. Matrices are stored by column as in Fortran. For 
example, the array A above can be thought of as a 1x6, 2x3, 3x2, or 6x1 matrix. 
If in a command, A is said to be 3x2, then TIMESLAB treats it as 


Iie 
PX mee PPA bs 
SG 
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B.4.2. Command Types 


The commands in TIMESLAB can be classified in terms of either their 
form or their utility. The structure of the program and the error messages it 
generates refer to the classification in terms of their form. 


Type 1 Commands 
These commands are of the form 
command name(list of arguments) 


For example, 


PLOT(x,n), TEXTCOLOR(O,2), and READ(air.dat,x,n) 


are type 1 commands. 
Type 2 Commands 


These commands consist of either an assignment statement such as n=10 
or operations such as z=x+y or z=x+10. The operations that are available are 
addition (+), subtraction (-), multiplication (*), division (/), exponentiation 
( * ), and negative exponentiation ( ~ —). These operations can be applied to 
integers, reals, and arrays, and several mixed type operations are available; for 
example, a real scalar c be subtracted from each element of the array X by the 
command Y=X-c. See Section B.4.3 for a complete description of the type 2 
commands. 


Type 3 Commands 
These commands are of the form 
variable name=command name(list of arguments) 


For example, the commands 
x=WN(seed,n) and y=SORT(x,n) 


are type 3 commands. 
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Type 4 Commands 
Examples of these commands are 
x=<1,2,3,4,5> and x=<x,5.43,z,y,6.2>, 


where any number of scalars or arrays can be concatenated. This is how 
vertical and horizontal concatenation of matrices is done. For example if 
A=<1,2,3,4,5,6>, B=<11,12,13,14>, and C=<A,B>, and one of the matrix 
commands (e.g., MMULT) is told that C is a (2x5) matrix, it will treat it as 


Pm i Sd it Bit 
all 2a<4 mG oe nae 


However, if A and B are treated as 


a 
ACs |e and B= | 


11 1 
33°6 


12 14 


and it is desired to vertically concatenate them, one would have to do the 
following steps: 

A=TRANS(A,3,2) 

B=TRANS(B,2,2) 

C=<A ,B> 

C=TRANS(C,2,5) 


Type 5 Commands 

These commands have no arguments and are given by merely entering 
their name. The HELP command with no argument is a type 5 command, while 
the HELP command with an argument is a type 1 command. 
B.4.3. Simple Assignments and Arithmetic Operations 

The type 2 command in TIMESLAB is used to do assignments and arith- 
metic operations. The general form of the assignment statement can be ex- 
pressed as (the equal sign can be preceded by a colon also) 

out=e 


where: 


out - Either a variable name or an element of an array (x[i] for 
example) which will receive the result of the assignment. 
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- Either an immediate integer, real, or character, a variable 
name, or an element of an array. The presence or absence 
of a decimal point determines whether an immediate value is 
taken to be real or integer. 


Some examples include (the code including semicolons and after are com- 


ments): 
n=1 ;integer since no decimal point 
c=3. ;real 
x=<1,2,3,4> jan array (this is a Type 4 command, not a Type 2) 
y=x ;assign the array x to the array y 
fname=’air.dat’ ;character variable 
fni=’wntn#? ;insert the value of a variable into a character 
x[(2]=x[3] ;operate on elements of an array 
n:=c ;convert the real c to an integer n 


The following rules are used to determine how the assignment is done: 


1. If there is no colon preceding the equal sign, then: 


a. 


If the left-hand side is the element of an array, then the value of the 
right-hand side is assigned to that element of the array unless the 
right-hand side is itself an array in which case the command aborts. 


. If the left-hand side is not an element of an array, then a variable 


having name specified by out is created having the same type and 
value as the right-hand side. 


2. If there is a colon preceding the equal sign, then: 


a. 


If the left-hand side is an element of an array, then the right-hand 
side is truncated and assigned to the specified element of the array 
unless the right-hand side is itself an array in which case the command 
aborts. 


. If the right-hand side is an integer, then a real variable is formed 


containing its value unless the left-hand side is an element of an array 
in which case it receives the value of the right-hand side. 


. If the right-hand side is a real, then its value is truncated and an 


integer variable is formed unless the left-hand side is an element of an 
array. 
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Arithmetic Operations 


The arithmetic operations that can be performed in TIMESLAB can be 
expressed in the general form (the equal sign can be preceded by a colon) 


out=e) Ge2 @ -- - @ex 


where: 
out - Either a variable name or an element of an array (x[i] for 
example) which will receive the result of the operation. 
e; - Hither an immediate value, a variable name, an element of an 
array, or an arithmetic operation itself within curly brackets 
({x+y} for example). 
® - One of the operators + ,— ,*,/,~° , or “— for addition, sub- 


traction, multiplication, division, exponentiation, and nega- 
tive exponentiation. 


If the equal sign is preceded by a colon, this indicates that the user wants 
to either truncate or change the type of the right-hand side before assigning 
its value to the variable or array element specified by out (see below). Some 
examples of arithmetic operations are: 


j=2 ;operations on integers 

k=3 

n=2*j+k 

X=LINE(2015=1701),.01) ;this is -1)= 99562 .480 js 98-99 0821.00 
x=44x j;now -4 to 4 


fx=exp~{-x~2/2}/{2*pi}~.5 ;this is normal pdf at -4 to 4 


How Expressions Are Evaluated 


When TIMESLAB encounters an arithmetic operation of the above form, 
it first evaluates the right-hand side, including determining its type (integer, 
real, or array) and value, and then places the result into the variable or array 
element specified by out. It first evaluates any expressions that are within 
curly brackets (working from left to right and always doing the innermost 
pair of brackets first if there are any nested brackets) and creates variables 
containing their value. Note that any variables created by such evaluations 
are deleted before the execution of the entire command is finished. This means 
that the user never sees these variables—for example they would not appear on 
the lists of variables if the INFO command were issued. If creating one of these 
variables would cause either the 60 variables of one type limit or the 10,000 
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array elements limit, TIMESLAB issues an error message to this effect, and 
the command aborts. 


To evaluate an expression within curly brackets or to evaluate the overall 
expression once all bracketed expressions have been evaluated, TIMESLAB 
uses the following rules: 


1. All exponentiations and negative exponentiations are evaluated, then all 
multiplications and divisions are evaluated, and finally all additions and 
subtractions are evaluated. In each case, the first operation from the left 
within the three types is the one evaluated next. 


2. The result of each such binary operation is placed into a temporary variable 
of a type determined by the type of the two quantities being operated upon. 
A quantity can be either an immediate or variable integer, an immediate or 
variable real, or an array. In this case TIMESLAB determines whether an 
immediate value is real or integer by the presence or absence of a decimal 
point. The type of the result of a binary operation is then determined by 
the following rules: 


a. If the two quantities are both integers, then the result is taken to be 
an integer unless the operation is to raise an integer to a negative 
power, in which case the result is taken to be real. 


b. If the two quantities are both reals, then the result is taken to be real. 


c. If one of the quantities is real and the other is an integer, then the 
result is taken to be real. 


d. If both quantities are arrays, then the operation is applied on corre- 
sponding elements of the two arrays, and the result is taken to be an 
array of length equal to the shorter of the two arrays if they are not 
of the same length. 


e. If one of the quantities is an array and the other is either an integer 
or a real, then the result is taken to be an array. 


Once the type and value of the right-hand side of the equal sign are deter- 
mined, then the same rules that were used in the simple assignment statement 
form of the type 2 command are used to do the assignment part of the arith- 
metic operation form. 
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Numerical and Syntax Errors 


The following numerical errors will cause a command to abort with a 


message being displayed: 


t 


2. 


Division by zero. 


An integer or real zero to a negative power. 


. A negative real (including an element of an array) to a real power. 


. An operation on reals that results in a value that is greater in absolute 


value than the largest four-byte real number (approximately 103°). 


. An operation on integers that results in a value that is greater in absolute 


value than 32,767. 


The following conditions will cause a command to abort with the message 


‘Syntax Error or Undefined Term’ being displayed: 


ile 


as 


An undefined variable being used on the right-hand side of the equal sign. 


A quantity (either an immediate value or variable name) more than 15 
characters long. 


. An illegal element of an array being used, for example, x[i] where i is 


undefined or less than 1 or greater than the length of x. 


. Two operations (other than “—) occurring next to each other. Thus c=- 


2.*-3. is illegal while c=-2.+*{-3.} is legal. 


. An unmatched curly bracket. 
. Nothing to the left of the equal sign. 


. Trying to assign an array to an element of an array. 


B.4.4. Listing and Deleting Variables 


TIMESLAB has room for 60 of each of integer, real, and character variables 


as well as for 60 arrays having a total of 10,000 elements. Each variable or array 
has a name attached to it. Each array also has a label having as many as 40 
characters. To get a list of all variables and arrays, one need only enter the 
command INFO. This command also gives the length of each defined array, 
its label, and how many free elements are left of the original 10,000 elements. 
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The various forms of the LIST command can be used to display the values of 
variables that have been defined, while the various forms of the CLEAN command 
can be used to delete variables that have been defined. 


B.4.5. Cases and Types 


Command names and DOS file names and device identifiers can be given in 
either upper or lowercase while all other names are case sensitive; for example, 
if an array called x has been defined, then TIMESLAB won’t recognize the 
array X in PLOT(X,n). This means that one can use the same name in upper 
and lower case for two different variables if desired. 

Note that the same name cannot be given to variables of different type. 

The only time that a “whole number” real immediate value (e.g., 23.) 
need have the decimal point attached is in a type 2 assignment statement. For 
example, the command x=10 will cause TIMESLAB to form an integer variable 
called x having the value 10. On the other hand, any time a real number is 
expected as an input argument in a command (as in seed in the command 
X=WN(seed,n)), and the desired value is a whole real number, the decimal 
point need not be included. Thus X=WN(12345,n) is a valid command even 
though the WN command expects a real scalar as the first argument. 


B.4.6. Error Checking 


TIMESLAB does extensive error checking. When a command is received, 
it first attempts to determine its type. If the command fits none of the five 
types, an “unrecognizable command type” message is displayed and the prompt 
reappears. The most common way this happens is for the user to leave off a 
parenthesis. 

Once the command type has been determined, the command name is 
checked against an internal list of available commands of that type. If it doesn’t 
exist, a message to that effect is displayed and the prompt reappears. For types 
2 and 4 this doesn’t apply since there is no command name associated with 
these types. 

Once TIMESLAB determines that a legitimate command has been issued, 
it checks any input arguments for having been defined. If a needed argument 
hasn’t been defined, the program issues an appropriate message and reissues 
the prompt. This often happens because a variable name has been misspelled. 
Again, any argument can have at most 15 characters, there can be at most 21 
arguments, and a command can have at most 72 characters. 

TIMESLAB also checks many arguments for being in legal ranges; for 
example, in PLOT(X,n), TIMESLAB checks whether previously an array X 
having at least n elements has been defined. 

Whenever TIMESLAB receives a command that requires it to use some 
of the computer’s memory as a workspace, it checks to make sure that enough 
workspace is available. An example of this is given by the command 
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G=TOEPL(rho,RO,n). 


This command is used to form an (nxn) Toeplitz matrix. TIMESLAB forms 
the matrix in a workspace and then copies it into the area of memory where 
arrays are stored. The workspace that TIMESLAB uses is restricted to 10,000 
elements. Thus if a user were to inadvertently try to form a 200x200 matrix, 
TIMESLAB would issue a message that there is not enough workspace and the 
command would abort. 


Whenever an error is encountered during the execution of a macro, the 
macro aborts (unless the ABORTOFF command has been issued), and the TIMES- 
LAB prompt is reissued. If the error can be easily fixed (for example by defining 
a variable that the macro needs), the macro can be restarted by issuing the 


command MACRO with no argument. For more information on this see Section 
B.7. 


The error checking has been done to try to make sure first that TIMESLAB 
doesn’t “bomb” (i.e., doesn’t encounter an error that causes an immediate 
return to DOS, or what’s worse, requires the system to be rebooted), and 
second that the user will be alerted to the existence of improper commands as 
much as possible. 


B.4.7. On-line and User-Definable Help Facilities 


Much of the content of this manual is available through the command HELP. 
Further, users can define their own help facilities via the NOTES command. 


On-line Help 


Issuing the command HELP results in a list of commands and a set of 
area names being placed on the screen. To get a brief description of what a 
command does and a list of its arguments (if it has any) one can issue the HELP 
command with the command name in parentheses. To get a list of commands 
that are available in one of the listed areas, the HELP command with the name 
of the area in parentheses can be issued. 


Special help screens are available while within either the TIMESLAB edi- 
tor or the graphics locator/editor FIND. The method of accessing these facilities 
is indicated on the screen whenever FIND or EDIT is being used. Also within 
FIND, one can get a list of the 256 ASCII characters and their ASCII codes. 
This same list is available from the TIMESLAB prompt by entering the com- 
mand HELP (ASCII). 


The usual HELP(topic) command can also be used from within EDIT by 
first striking Alt-F1. Thus while using EDIT to create a macro file one can 
recall what the arguments are for a particular command. 


SEC B.4 OVERVIEW OF CAPABILITIES 431 


User-Definable Help 


A user can provide further help facilities by first forming a file called a 
notes file using EDIT, and then using the NOTES command to retrieve the notes 
that have been defined. This method can be used to keep track of the names 
and locations of macro or data files, or to store information about an analysis 
that has been performed. A notes file consists of a text file (creatable via the 
TIMESLAB editor or any other text editor that can create a pure ASCII file) 
containing a series of what TIMESLAB calls notes. A note consists of a first 
line containing a note name (as many as 15 characters starting in column one 
having no spaces) followed by as many lines (having as many as 72 characters 
each) containing the note. Each of these note lines must have a blank character 
in column one. 

Once a notes file has been created, entering the NOTES command with the 
name of a file in parentheses results in the display of the list of note names 
in that file. Entering NOTES(filename,notename) results in a display of that 
note. 


B.4.8. Creating and Saving Arrays 


As we mentioned above, the basic objects of interest in TIMESLAB are 
arrays. There are two methods for creating arrays: (1) reading them from a 
disk file (via the READ command), and (2) as output from a command. Once 
an array has been defined, it can be saved to a disk file by the command 
SAVE(X,n,fname), where fname is the name of the file to receive the array 
X. Unless an array is saved to disk in this way, it is lost when the current 
TIMESLAB session ends. 

The user can form a file on disk containing an array using any text editor 
(including the EDIT command) that is capable of producing a pure ASCII file. 
The TIMESLAB editor is specially designed to form and modify the short 
data and macro files that TIMESLAB uses. Getting in and out of EDIT is 
done quickly and easily. If some other editor is used, care must be taken that 
the result contains no non-ASCII characters as the READ command will not 
recognize them. This includes for example the tab character used by many 
editors. 


TIMESLAB Data Files 


TIMESLAB will accept data files of two kinds: (1) formatted or (2) free 
formatted (the SAVE command saves an array to disk in formatted form). For 
either kind the first line of the file must be a label identifying the array. This 
label can have as many as 40 characters. Once TIMESLAB has read the file 
this label is used as a heading for lists and plots of this array. 

In a formatted file, the second line of the file must contain the number of 
elements in the array followed by a Fortran FORMAT statement (including its 
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parentheses) describing the format that the elements of the array have been 
entered in the file. The array length and format can be anywhere on this second 
line as long as (1) the length comes first, (2) there is at least one space between 
it and the format statement, and (3) the format statement has no more than 
30 characters. The format statement must include the beginning and ending 
parentheses. 


In an unformatted file, the second line contains only the length of the 
array. After this second line the elements of the array are entered in any 
format desired as long as there is at least one space separating consecutive 
elements. In a formatted file, the elements of the array must follow the second 
line and be entered according to the format given on the second line. This 
ability to have formatted files allows the user to have identifying information 
in the file or to skip columns of numbers in the file if desired. 


Note that the SAVE command makes it possible to append the array being 
saved onto the end of an existing file, and that the READ command allows one 
to skip several data sets on a file before reading the one that is desired. 

To illustrate data files, we include a listing of the file AIR.DAT which 
consists of a formatted data file and is Series VI in Chapter 1. Using the 
formatted form for AIR.DAT allows one to put the year into the file. The 
format statement tells TIMESLAB to skip the year when reading the data. 


Airline Passengers, 1949-60 
144 (5X,12F5.0) 

1949 112 118 132 129 121 135 148 148 136 119 104 118 
1950 115 126 141 135 125 149 170 170 158 133 114 140 
1951 145 150 178 163 172 178 199 199 184 162 146 166 
1952 171 180 193 181 183 218 230 242 209 191 172 194 
1953. 196 196 236 235 229 243 264 272 237 211 180 201 
1954 204 188 235 227 234 264 302 293 259 229 203 229 
1955 242 233 267 269 270 315 364 347 312 274 237 278 
1956 284 277 317 313 318 374 413 405 355 306 271 306 
1957 315 301 356 348 355 422 465 467 404 347 305 336 
1958 340 318 362 348 363 435 491 505 404 359 310 337 
1959 360 342 406 396 420 472 548 559 463 407 362 405 
1960 417 391 419 461 472 535 622 606 508 461 390 432 


There are two other points to be made about data files. First, several data 
sets can be placed in the same file. In this case, each one must be in the form 
described above. Then the four argument form of the READ command can be 
used to read them one at a time (see the REG macro for an example of how 
TIMESLAB can be tricked into reading a data file that has several data sets 
stored by column). Second, a data point that is represented in the data file 
by just a decimal point will be read as a zero. Thus one can represent missing 
values by a decimal point. 
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B.4.9. Overwriting Previously Defined Variables 


When TIMESLAB is first started, it is in what is called the “overon” 
mode; that is, if the user enters a command that has as output a variable that 
has already been defined, TIMESLAB will not ask permission to overwrite the 
new value onto the old. One can issue the type 5 command OVEROFF to tell 
TIMESLAB to henceforth ask for permission to overwrite a variable any time 
that a command would result in such overwriting. The command OVERON puts 
TIMESLAB back into the overon mode. If a user would like to always be in the 
overoff mode, the OVEROFF command can be included in the AUTOEXEC.MAC 
macro file (see Section B.4.14). 


B.4.10. Printed Output 


TIMESLAB can produce printed output in several ways. See Section B.5 
for a description of how to obtain hard-copy versions of graphics screens, and 
see the descriptions of the various printing commands in Sections C.1 and C.2. 


B.4.11. The DOS Intraline Editing Functions 


The DOS command line special intraline editing functions work when at 
the TIMESLAB prompt in the same way as at the DOS prompt. Thus when at 
the ? prompt (in the text mode only), pressing the F3 function key brings back 
the last command that was entered, pressing F1 repeatedly brings back one 
character at a time from the previous command, pressing F2 and a character 
x copies all characters from the previous command up to but not including 
x, and pressing F4 and a character x followed by F3 brings back the previous 
command with the characters up to x deleted. Characters can be inserted into 
the previous command by pressing F2 followed by the character before where 
the insertion is to be made, pressing the Ins key, typing the desired insertions, 
and then pressing F3 to bring back the rest of the line. Characters can be 
deleted in the previous command by a similar process except using the Del key 
to delete the desired characters. Using these special functions one can: 


1. Instantly repeat a command by pressing F3 and return. 
2. Edit and reenter a command without having to retype the whole line. 


3. Form a new command that is similar to the previous command. 


The DOS manual has more details about these editing functions. 
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B.4.12. The TIMESLAB DOS Mode 


TIMESLAB has a special type 5 command called DOS. Entering this com- 
mand results in the prompt ?d> where d is the current default disk drive. 
At this point TIMESLAB is essentially in a “DOS mode,” wherein any DOS 
command can be issued, including invoking any text editor or copying files. 
To exit this DOS mode, one answers the prompt with the word exit in either 
upper or lowercase. This returns control to the usual TIMESLAB prompt. 

To use the DOS command, TIMESLAB must be able to access a copy 
of COMMAND.COM either by having one in the same (sub) directory as 
TIMESLAB.EXE or by having a PATH to it (see the DOS manual). If TIMES- 
LAB can’t find COMMAND.COM or the command issued at the DOS prompt 
would result in not having enough RAM, TIMESLAB will issue a message to 
that effect. For example, TIMESLAB uses approximately 480K of RAM and 
DOS uses anywhere from around 20K for Version 2.0 to around 60K for Ver- 
sion 3.2. Thus if one has Version 3.2 of DOS, only programs requiring less 
than around 100K RAM could be executed from the TIMESLAB DOS prompt 
(480 + 60 + 100 = 640). 


B.4.13. The TIMESLAB Full Screen Text Editor 


A full screen editor specially designed for the relatively short data and 
macro files used by TIMESLAB can be invoked by issuing the EDIT command 
(see Section B.6). This editor uses a subset of the features of the IBM Profes- 
sional Editor. 


B.4.14. AUTOEXEC.MAC 


If there is a file called AUTOEXEC.MAC on the default disk drive when 
TIMESLAB is invoked, then TIMESLAB will execute it as a macro. This 
ability can be used in a similar way as AUTOEXEC.BAT in DOS. 


B.4.15. Random Number Generator Seeds 


Several of the TIMESLAB commands generate random numbers and thus 
require a “seed” (a real scalar used to start the random number generator). 
The user can either choose to specify a seed, in which case an experiment can 
be rerun at a later time with the same results if the same seed is used, or choose 
to use a seed that TIMESLAB will generate. To get this default seed, the user 
need only enter 0. as the argument in the command being used. Any time one 
of the simulation commands is issued, TIMESLAB generates an output seed 
that is stored internally. The default seed for a command is in fact this output 
seed from the last time one of the simulation commands was used, unless the 
present command is the first time a simulation command has been issued, in 
which case TIMESLAB will find a default seed by reading the internal clock 
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on the PC. 
B.4.16. Keeping a Record of a Session 
The command 
RECORD (fname) 
followed later by the command 
RECORD (close) 


will keep a record of the results displayed by several commands (see the de- 
scription of the RECORD command in Section C.2). The RECORD command also 
keeps a record on the file of every command that is entered from the keyboard. 
This feature makes it possible to later create a macro by editing the file of 
commands. 

If the filename is one of the valid names for the printer (for example, Ipt1 
or prn), then the record is directed to the printer. 


B.4.17. Exiting from TIMESLAB 


To exit from TIMESLAB and return to DOS, one need only answer the 
prompt with the type 5 command QUIT. Any arrays that have not been saved 
onto a disk file either by the SAVE command or by using the TIMESLAB 
editor will of course not be available the next time TIMESLAB is used. 


B.5. TIMESLAB Graphics 


The TIMESLAB program has been designed with the graphical nature of 
time series analysis in mind. The commands that produce high resolution pixel 
graphs on the screen are: 


HIST - Simple histogram plot. 

INFQNT - Produce an informative quantile plot of a data set. 
PLOT - Simple Y versus X plot. 

PLOT2 - Superimpose the plot of two Y variables versus the 


same X variable. 


PLOTK - Superimpose the plot of as many Y variables as de- 
sired, each versus its own X variable. 


PLOTSP - Spectral density plot (log scale). 
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PLOTCSP - Spectral density plot (nonlog scale). 


RESCREEN - Reproduce a graph that was previously saved to a file 
using the SAVESC command. 


In this section we give detailed information about how these commands 
and others can be used to provide interesting graphical displays for studying 
time series analysis. 


B.5.1. Specifications for Plots 


The graphics routines listed above all produce plots consisting of a hori- 
zontal and vertical axis (with numerically labeled tic marks on both axes), one 
or more graphs on the axes, and one or more captions above the plots. Because 
of the lack of resolution on the screen, there are no descriptive labels below 
the horizontal axis or to the left of the vertical axis. Such labels can be added 
to graphs by the user using the FIND utility (see below) or the two- or three- 
argument forms of the LABEL command (see the HAT2 macro for an example of 
using the LABEL command). 

The following values determining the size and location of plots can be 
specified by the user using the PLOTSIZE command: 


npixx,npixy 


Length in pixels of the horizontal and vertical axes 
(defaults are 480 and 120). 


nxi,ny1 - Pixel location of the origin (lower left-hand corner of 
the screen is (0,0), defaults are 55 and 30). 


Number of tic marks on the horizontal and vertical 
axes (defaults are 10 and 10). 


nticsx,nticsy 


nclabx,ndecx - Numerical labels at the tic marks on the horizontal 
axis have a total of nlabx characters (including the 
decimal point and any minus sign) with ndecx places 
to the right of the decimal point (defaults are 8 and 
2h 


nclaby,ndecy - Numerical labels at the tic marks on the vertical axis 
have a total of nlaby characters (including the decimal 
point and any minus sign) with ndecy places to the 
right of the decimal point (defaults are 6 and 2). 


Note that each character in a tic mark label is made up from a pattern 
of 8 x 8 pixels being on or off. There is no space between the 8 x 8 pixel 
regions as the spacing is supplied by columns in the characters being off. The 


SEC B.5 GRAPHICS 437 


bottoms of the tic mark labels on the horizontal axis are placed 12 pixels below 
the axis. In addition, one should reserve the bottom eight pixels of the screen 
for the “graphics menu” (see below). Thus ny1 should be at least 20. All 
of the graphics commands leave a space of two pixels between the end of the 
vertical axis tic mark labels and the vertical axis. Thus nx1 should be at least 
8*nclabx+2. 


B.5.2. Graphics “Windows” 


By default, TIMESLAB produces plots that fill much of the PC’s screen, 
and unless the PLOTON command has been issued, any plot that is produced 
is erased before another can be produced. It is possible (by using the PLOTON 
and PLOTSIZE commands) to divide the screen into different areas and produce 
graphs in more than one of them at the same time. These areas are called 
windows although they are not strictly windows in that commands can produce 
graphics that intrude upon more than one area of the screen. The following 
strategy can be used to do windowing: 


1. The PLOTON command is used to tell TIMESLAB not to switch back to 
the text mode between commands. The prompt will now be reissued at 
the top of the screen after each command so that it will not overwrite 
any graphs already on the screen. Any command that results in displayed 
output may overwrite graphs that are on the screen. There is no separate 
window for displaying nongraphical material. 


2. The PLOTSIZE command can be used to specify the size and location of 
subsequently produced plots. Users can make these specifications anything 
that they want, but issuing the command PLOTSIZE(k) for the values of 
k in the table below sets up certain standard windows. The values in the 
table have been designed so that different regions of the screen will not 
interfere with each other. For example, regions 1 through 4 do not overlap 
unless the user has increased nclabx or nclaby from the default values. 


This strategy allows two important uses of graphics: (1) several graphs can 
be established on the same screen (see the COS macro for an example), and (2) 
many graphs can be superimposed on the same axes (see the HAT2 macro for 
an example). The BATCHON command is important in this strategy. Ordinarily, 
the user wants the graphics menu to appear after producing a plot so that it 
can be printed or saved to a file. Unfortunately, this means that the user must 
strike a key in between each graph. If the BATCHON command is issued, then 
there is no pause between plots. Thus before the last plot in a series of plots 
is formed, one might want to issue BATCHOFF so that the graphics menu will 
appear after all of the plots are on the screen. One can also use the GRMENU 
command to place the graphics menu on the screen at any time that the screen 
is in the graphics mode. 


438 USER’S GUIDE APP. B. 


Standard Windows Using the PLOTSIZE(k) Command 


k Region of Screen npixx npixy nxi nyi 
0 Full screen 480 120 55 20 
fo Upper left 260 60 50 120 
7 Lower left 260 60 50 20 
3 Upper right 260 60 360 120 
4 Lower right 260 60 50 20 
5 Left half 260 120 50 20 
6 Right half 260 120 360 20 
i Top half 480 60 50 110 
8 Bottom half 480 60 50 20 


B.5.3. The Graphics Menu 


Whenever TIMESLAB produces a graph on the screen (unless the BATCHON 
command has been issued), it places a message (called the graphics menu) at 
the bottom of the screen (in the bottom eight rows of pixels) alerting users to 
the fact that they can perform several useful operations, and then pauses until 
users either perform one of these operations or indicate that they want to go 
on to the next command. It is also possible to place this graphics menu on the 
screen at any time the screen is in the graphics mode by issuing the GRMENU 
command. 

The operations that are available from the graphics menu are determined 
by striking one of the following function keys (striking any other key will result 
in the TIMESLAB prompt being reissued): 


Fi - This invokes the FIND utility; that is, an interactive locating 
and labeling facility becomes available (see Section B.5.5). 


F3 - This causes whatever is currently on the screen (with the 
graphics menu erased) to be printed on whatever printer is 
attached to the PC. In order for this to operate properly, the 
PRINTSEL command has to have been issued previously to 
tell TIMESLAB what type of printer has been attached. See 
Section B.5.4 for more details about obtaining such screen 
dumps. 


FS - This causes TIMESLAB to create a disk file (it will ask the 
user for the name of the file to be formed) containing a com- 
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pressed version of what is currently on the screen, again with 
the graphics menu erased. This compressed screen image can 
be instantly redisplayed later using the RESCREEN command. 


F9 - This allows the user to break out of a macro if one is being 
executed. 


If the BATCHON command has been issued, then the graphics menu is not 
displayed and TIMESLAB does not pause to allow the user to see the com- 
pleted graph. If the SAVESC and/or PSON commands have been issued, then the 
completed graph will be saved to a file or dumped to the printer before the next 
command is executed. See the COS macro for an example of how selectively 
issuing the BATCHON and BATCHOFF commands can be used to obtain several 
plots on the screen with no pause until after the last one is formed. 


B.5.4. Graphics Screen Dumps 


Any time that a graph is on the screen, a pixel-by-pixel image of the 
screen can be produced on the printer that is attached to the PC as long as 
two requirements are met: 


1. The printer is capable of performing high resolution graphics. Examples 
of this are: (a) the IBM Graphics Printer or any printer that claims to 
be compatible with it (these are 9 pin dot matrix printers), (b) a Toshiba 
321 24 pin dot matrix printer or any printer that claims to be compatible 
with it, and (c) the Hewlett-Packard Laserjet Printer or any laser printer 
that is compatible with it. 


2. The PC has been alerted to the fact that screen dumps will be requested. 


The manner in which the second requirement is met depends on whether 
or not the printer attached to the PC is one of the types for which TIMESLAB 
has a built-in screen dump capability. These types are the three that are listed 
in item 1 above. If the attached printer is one of these three types, then one 
need only use the PRINTSEL command to tell TIMESLAB which type it is, 
and then graphics screen dumps can be obtained in either of the following two 
ways: 


1. By issuing the PSON command prior to producing the graph. This method 
will produce a screen dump for any graphics screen that is produced until 
the command PSOFF is issued and is included in TIMESLAB primarily for 
use within macros. 


2. By striking F3 from the graphics menu. 
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If the PC is attached to a laser printer, there are a few other considerations 
involved in producing screen dumps. First, communication between the printer 
and the PC must have been established. The manual for the printer should 
contain information about how this is done. Second, the image constructed on 
the printer is approximately 4.25 inches wide and 2.5 inches high. Thus only 
three such images will fit on one page. After three images are produced, issuing 
the command PAGE will cause the page to be ejected. The third consideration 
with laser printers is how much “on-board” memory the printer has. For exam- 
ple, the H-P Laserjet Printer has only enough memory to produce one image 
on a page, while the Laserjet Plus can produce more than the three which will 
fit. 


Screen Dumps on a Nonstandard Printer 


If the printer that is attached to the PC is not one of the three types 
described above (this can only really be determined by trying the methods 
described above), then the user will have to rely on the ability of DOS itself to 
produce graphics screen dumps (on the standard printers, TIMESLAB does the 
screen dumps). In this case, the screen dump is produced by striking the Shift 
and PrtSc keys at the same time while the graph is on the screen. When this 
is done, DOS takes control from TIMESLAB and tries to do a screen dump. 
Unfortunately, screen dump software is not a standard part of DOS. Thus the 
user must have access to such software for their particular combination of PC 
and printer. For example, issuing the DOS command GRAPHICS prior to 
invoking TIMESLAB will make it possible to do screen dumps to the IBM 
Graphics Printer via the Shift-PrtSc method. In general, the user must have 
software analogous to GRAPHICS.COM for their PC-printer combination, and 
have issued it prior to entering TIMESLAB. 


B.5.5. The FIND Utility 


Striking F1 from the graphics menu invokes what is called the FIND utility. 
This utility sets up a graphics cursor on the screen that can be moved to any 
position on the screen by using the keys on the numeric keypad. The z and 
y values corresponding to the pixel at the lower left-hand corner of the cursor 
are displayed in the upper right-hand corner of the screen. 


Locating Points on a Graphics Screen 


When FIND is invoked, striking any of the keys labeled 1, 2, 3, 4, 6, 7, 8, 9 
on the numeric keypad moves the cursor 10 pixels in the direction indicated by 
the key. For example, striking the key labeled 7 on the numeric keypad causes 
the graphics cursor to move up 10 pixels and 10 pixels to the left. The distance 
traveled by the cursor can be changed to any one of 1 through 10 pixels by 
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striking the corresponding function key. As the cursor moves, the z and y 
values are updated. Large moves of the cursor are also available by striking 
combinations of the Ctrl key and numeric keypad keys. For example, Ctrl-right 
arrow moves the cursor all the way to the right, Ctrl-PgDn moves it all the 
way to the bottom, and so on. The cursor can be moved directly to a specified 
(x,y) coordinate by striking Ctrl-F1. Note that all possible key strokes are 
described on the FIND help screen that is available by striking Alt-F1 while in 
FIND. 


Labeling Graphs 


While in FIND, the user can place any of the 256 ASCII characters any- 
where on the screen. Some of these characters can be entered by just striking 
their key on the keyboard. To enter a character that doesn’t have a key, one 
enters its ASCII code while holding down the Alt key. When the Alt key is 
then released, the character will appear. A list of the ASCII codes for the 256 
available characters is available within FIND by first going to the FIND help 
screen (by striking Alt-F1) and then striking Fl. One then returns immedi- 
ately to the screen being edited by striking any key. Messages guiding the user 
through this process are placed on the screens involved. 


Other Features of FIND 


One can easily draw lines and rectangles on the screen by using com- 
binations of the Alt key and function keys. The FIND help screen contains 
instructions on how to do this. Striking Ctrl-F9 while in FIND results in a 
blank screen that can be used to construct screens (from the characters and 
rectangles) that can later be printed or stored. Also while in FIND, Alt-F2 is 
a toggle for turning on and off the display of the current (z,y) values, and 
Alt-F3 is a toggle for turning the cursor “on and off.” Actually Alt-F3 toggles 
whether the cursor is in the foreground or background color. This is important 
when placing the cursor in a solid rectangle wherein it would be invisible if one 
didn’t have the ability to make it the background color. 


There is one kind of graphics screen where FIND operates a little differently. 
There is a TIMESLAB command called RESCREEN which places a screen image 
(that has been previously saved onto a file by either striking F5 at the graphics 
menu or using the SAVESC command) onto the screen. When FIND is invoked 
after RESCREEN, TIMESLAB doesn’t have any way of knowing the real-world 
environment of the screen. Thus there is no display of (z,y) as the cursor 
moves. 
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Exiting from FIND 


To exit from FIND, one strikes Alt-F 10. This puts users back into a situa- 
tion where they can either print the screen, save it for later retrieval, or reenter 
FIND. At this point, striking any key other than F1, F3, or F5 gets back to the 
TIMESLAB prompt. 


B.5.6. Saving and Retrieving Graphics Screen Images 


Some very effective graphics screens (and thus hard copies of screens) 
can be constructed by using the ability of TIMESLAB to store and retrieve 
files containing compressed graphics screens. A typical graph produced by 
TIMESLAB can be stored into a file of approximately 3000 to 4000 bytes. 
Thus 90 or 100 such screens can fit on one floppy. There are two ways to save 
a graphics screen to a file: 


1. By striking F5 at the graphics menu. TIMESLAB then asks for the name 
of the file to receive the compressed screen image. 


2. By issuing the TIMESLAB command SAVESC prior to issuing the command 
that places the graph on the screen. SAVESC has one argument, namely 
the name of the file to receive the image. 


Once such files are formed, one can then “rescreen” one or more of them 
by issuing the RESCREEN command from the TIMESLAB prompt. To rescreen 
one image, the command is RESCREEN(fname) where fname is the name of the 
file containing the image. To rescreen several images, the RESCREEN command 
can be issued with the names of the files containing the screens (in the desired 
order) as arguments, together with a final argument that is 1 if each screen is 
to be erased before the next is placed on the screen, or 2 if the screens are to be 
superimposed. If many images have been stored on files having names that all 
begin with the same set of characters and end with consecutive integers (e.g., 
wn3,wn4,wnS,wn6), then they can all be rescreened by issuing a command of 
the form RESCREEN(wn,3,6,iopt), where iopt=3 means erase in between and 
iopt=4 means don’t erase in between. Thus one can superimpose images or 
else just see a series of them one after another. This happens fast enough (the 
speed depends on the speed of disk accesses on the machine being used) that 
one can create animation type effects. If the images are stored on a floppy, 
there is perhaps a one-second pause between images while on a hard disk there 
is about half this pause. If the screen images are stored on a RAM disk, there 
is hardly any pause between rescreened images. Note that a macro can be used 
to create the files containing the images. 

Once the final image from a RESCREEN appears on the screen, the graphics 
menu is placed on the screen so that this (possibly new) image can be edited 
using FIND and then printed or saved. 
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B.5.7. Scaling and Labeling Graphs 


Some very important parts of the appearance of any of the graphs produced 
by TIMESLAB are the caption above the graph, the graph’s scale, and the 
numerical labels placed at the tic marks on the graphs. 


Caption and Axis Labels 


TIMESLAB places the label (or labels if appropriate) associated with the 
array (or arrays) being plotted above the graph. Because of the lack of reso- 
lution on the screen, there is no textual label placed to the left of the vertical 
axis or below the horizontal axis. The FIND utility can be used to add such 
labels if desired, or to place text anywhere on the screen. There are also two 
forms of the LABEL command which can be used to add labels to plots. 


Scaling 


Each of the PLOT, PLOT2, PLOTK, HIST, and PLOTCSP commands can be 
given in two forms. The first form (with no scaling information) tells TIMES- 
LAB to use the actual minimum and maximum values of what is being plotted 
as the values at the ends of the axes. The second form (with scaling informa- 
tion) tells TIMESLAB to use the minimum and maximum values entered as 
arguments as the values at the ends of the axes. If the second form is used 
and there is a value to be plotted that is outside of the user-specified range, 
TIMESLAB reverts to the first form. Typically then, one first uses the first 
form (unless there are some natural limits on the graphs such as —1 to 1 for 
a correlogram), inspects the resulting plot, and then reissues the command in 
the second form with some esthetically appealing limits. Being able to specify 
limits like this also allows several plots to be produced with the same scale. To 
illustrate these ideas consider Figure B.1 which consists of hard copies of the 
graphs produced by the commands 


x=WN (12345, 100) 
LABEL(x)=’PLOT(x, 100)’ 

PLOT (x, 100) 
LABEL(x)=’PLOT(x,100,0,100,-4,4)’ 
PLOT(x,100,0,100,-4,4) 


In this case we could anticipate what scaling information would lead to a 
pleasing graph. 
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1.0 20.88 48.68 60.48 80.20 100.0 


PLOT (x, 100, @, 108, -4, 4) 


A 20.00 0.0 60.00 60.0 100.0 


Figure B.1. The Effect of Using Scaling Information in the PLOT Command. 
Tic Mark Labels 


For the vertical axis in PLOTCSP and for both axes in PLOT, PLOT2, 
PLOTK, and HIST, the user can determine what the tic mark labels are. TIMES- 
LAB attempts to put numerical labels at as many of these tic marks as possible. 
It will also always put labels at the intersection of the two axes. If a numerical 
label is too large to fit the format specified by nclabx, ndecx, nclaby, and 
ndecy, then a string of asterisks is placed at the tic mark instead. If this hap- 
pens, the user must rescale the array or arrays being plotted so that they will 
fit. This can be annoying but rarely happens and it is easy to do the rescaling. 

To illustrate how appealing labels at the tic marks can be obtained, con- 
sider how the horizontal axis is labeled. The minimum value (either the actual 
minimum for the first form or the inputted value for the second form) is placed 
at the intersection of axes. The label at the first tic mark would then be the 
minimum value plus a fraction of the range of the axis (the maximum value 
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minus the minimum value) equal to the number of tic marks on that axis. 
TIMESLAB tries to place this label at the tic mark with the decimal point 
directly under the mark. If the label would not fit without interfering with the 
one already positioned, TIMESLAB goes on to the next tic mark and repeats 
the process except that the label would be the minimum value plus a fraction 
of the range equal to two over the number of tic marks. The vertical axis is 
treated in exactly the same way. 

As an example of producing nice tic mark labels, suppose the array being 
plotted on the horizontal axis consists of the numbers 1, 2, ... ,n. In this case, 
the user most likely will want the tic mark labels to be whole numbers. Using 
zero and the next multiple of ten greater than or equal to n as the endpoints 
of the axis will give such labels if one is using ten tic marks. Note that such a 
strategy can be used to move a graph away from the axes as well. 


B.6. The TIMESLAB Text Editor 


During a TIMESLAB session one often needs to create or modify macro 
or data files. The EDIT command allows this to be done without going to the 
trouble of leaving TIMESLAB or going to its DOS mode in order to use an 
external text editor. EDIT is a simple full screen editor; that is, a portion of 
a file is placed on the screen and changes can be made directly on the screen. 
Thus the user positions the cursor to where changes are to be made and then 
enters the changes directly by either replacing a character, inserting a character 
or a line, or deleting a character or a line. 

EDIT is not intended to have all of the features of a full screen editor such 
as the IBM Professional Editor. It has no block copies, moves, or deletions, or 
the ability to make global searches or replacements, or the ability to merge text 
from another file. Rather it is intended to be very simple to use to form and 
modify the kinds of files that TIMESLAB uses (particularly data and macro 
files). Files having as many as 2000 lines and 18,000 characters can be edited 
using EDIT. 


Entering and Exiting the Editor 
To edit an existing ASCII file, one need only enter the command 
EDIT(fnold,fnnew), 


where fnold and fnnew are the names of the existing file and the file to receive 
the modified version of the file. If the user wants these names to be the same, 
then the command can be issued as EDIT(fname). If there is no file on the 
default drive having this name, then EDIT will assume that a new file is to be 
created. 

To leave the editor and return to the TIMESLAB prompt, one has two 
choices: F3 which saves the file and returns to the TIMESLAB prompt, and 
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F4 which returns directly to the TIMESLAB prompt without saving the new 
file. Usually one uses F3 but sometimes it is helpful to start over with a fresh 
copy of the old file. Then one can use F4 and call EDIT agam. In either case, 
EDIT will ask for verification that the user genuinely wants to leave EDIT. 


Adding and Deleting Lines 


After entering the EDIT command the user sees an introductory screen that 
indicates whether an existing file is to be edited or a new file is to be created. 
If a new file is being created, lines marking the beginning and end of text will 
appear on the screen with the cursor at the first line. To begin entering text, 
one strikes F5 to get into the “insert lines mode”. Striking a carriage return at 
the end of a line causes EDIT to give a new blank line to use to enter the next 
line. When all lines have been entered, the only way to exit this insert lines 
mode is to strike another carriage return while a new line is still blank. 

If an existing file is being edited, the initial lines of that file will be placed 
on the screen and the cursor will be at the top line of the screen. Then the 
cursor can be moved to where changes are to be made, and the changes made. 
At any time in an editing session, one can insert lines by positioning the cursor 
to the line before where the lines are to be inserted, and then striking F5. This 
places EDIT into the insert lines mode, and again the only way to leave this 
mode is to strike a carriage return on a newly opened blank line. 

To delete a line, one need only position the cursor to the line to be deleted, 
and then strike F6. 


Positioning the Cursor 


One moves the cursor by using the keys on the numeric keypad together 
with the Del, Ins, Tab, and Ctrl keys. The keys on the numeric keypad will 
only work properly if the keyboard is in the Num Lock off state. If a number 
appears on the screen when one of the keys on the numeric keypad is struck, 
then the keyboard is in the Num Lock on state. If this happens, one need only 
strike the Num Lock key to reverse the state. 


The following keystrokes move the cursor to the indicated positions: 


Arrows - One position in the direction of the arrow. 
Ctrl - Seven positions to the right. 

Tab - Seven positions to the right. 

Ctrle - Seven positions to the left. 


FQ - To the left or right end of the current line. 
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Home - To the top of the screen. 
End - To the bottom of the screen. 
Pg Up - Up 12 lines. 

Pg Dn - Down 12 lines. 


Ctrl-Home - ‘To the beginning of the file. 


Ctrl-End - ‘To the end of the file. 


Deleting, Overwriting, and Inserting Characters 


To delete the character where the cursor is located, one need only strike 
the Del key. To delete the character to the left of the cursor, one strikes the 
backspace key. 


When the cursor is located at a certain character on the screen, two things 
can happen when one of the usual alphanumeric keys is struck: either the 
current character is replaced by the struck character or the struck character is 
inserted before the character where the cursor is. These two modes are called 
the overwrite and insert modes and which will happen is controlled by the Ins 
key. When EDIT begins it is in the overwrite mode. The Ins key is used to 
switch back and forth between the two modes. One can always tell which mode 
is in effect because the cursor is normal size in the overwrite mode and larger 
than normal when in the insert mode. 


HELP Within EDIT 


While in EDIT one can get a screen that gives directions on how to use the 
editor. This is done by striking Fl. By striking Alt-F1, one can also get the 
usual TIMESLAB HELP. This is particularly useful when forming a macro file. 
Thus if users forget the arguments of some command that they want to use in 
a macro, they can strike Alt-F1, at which point TIMESLAB asks which help 
topic users want information about. Once the information is on the screen, one 
returns to editing (right where they were) by striking any key. 


B.7. Using Macros 


The ability to execute macros makes TIMESLAB an interpreted program- 
ming language. The purpose of this section is to discuss in detail the features 
of TIMESLAB macros. In the next section we discuss several examples. 
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B.7.1. Basic Features 


A macro is a collection of TIMESLAB commands residing on a disk file. 
The user invokes the macro by entering the command MACRO(fname) where 
fname is the name of the file containing the commands to be executed. 


Example 


To illustrate some of the features of macros, we include the example below. 
This example is contained in the file EXAMPLE.MAC and we will refer to it 
as the EXAMPLE macro. Note that we have inserted line numbers in this and 
all other macros in the book. These line numbers are not and should not be 
included in the actual file. EXAMPLE does very few calculations and is only 
intended to illustrate the basic form and structure of macros. Note that this 
example contains a large number of comments, that is, lines that begin with one 
or two semicolons and lines that contain a semicolon followed by a comment. 


Lass 
2 ie EXAMPLE .MAC: This macro illustrates the basic features of 
BGA macros. All that it does is find the sum of the 
C A first n natural numbers for a user-specified value 
aes s of n. The value of n must be less than or equal 
Guys to 10. If you haven’t defined n, break the 
‘Tees macro now and define it. Then re-invoke the 
Sais macro by issuing the command MACRO with no 
Olmrs argument. 
10 ;; 
1 Te ari INPUT: n (an integer between 1 and 10, inclusive) 
12 35; 
L335 OUTPUT: sum (an integer containing the sum) 
4 ss 
15 PAUSE ;this lets user break and define n if necessary 
16 ; 
ipo by having the next line, the prolog of the macro can be skipped 
rs by entering the command MACRO(example,start) 
To 
20 ;start 
21 IF(n.1t.1,error) ;check if any error in value of n 
22 IF(n.gt.10,error) ;if yes, go to line labeled error 
23 GOTO(st1) ;if no, go to line labeled sti 
24 3 
25 ;error 
26; 
27 PROMPTON ;turn prompt back on so that error message will be seen 
28 35 
29 sere n must be between 1 and 10, inclusive 
30. 43% 
31 GOTO(end) ;now exit the macro because of error 
32 ; 
33 ;sti 
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Hi Now we will find the sum by five methods. 


Ae pause after each one. 
PAUSE 

;method1 

CLS ;clear screen 


Se Implied loop using IF and GOTO: 
mae This is moderately fast. 


331 

sum=sumti 
IF(i.eq.n,e1) 
i=it1 

GOTO(s1) 

;e1 

LIST (sum) 

PAUSE 

;method2 

CLS ;clear screen 


Ay Implied loop using three-argument IF: 
AH This is the fastest of the four loops. 


i=0 

sum=0 

i=it1 

sum=sumti 

nmi=n-i 

IF(nmi,-3,1,-3) ;nmi=0 means stop loop 

LIST(sum) 

PAUSE 

;method3 

CLS ;clear screen 

Mir IF...ENDIF construct: 

+3 

i=1 

sum=0 

382 

IF(i.le.n) 
sum=sumti 
i=it1 
GOTO(s2) 

ENDIF 

LIST (sum) 

PAUSE 

;method4 

CLS 


D221 


449 


450 USER’S GUIDE APP. B. 


Sac WHILE-END Construct: 

SOc" This is slow because it must back up in the file. 
90 5; 

91 i=1 

92 sum=0 

93 WHILE(i.le.n) 


94 sum=sumt+i 


95 i=iti 

96 END 

97 LIST(sum) 

98 PAUSE 

99 ;method5 

100 CLS ;clear screen 

(WO Bes 

102 "55 Fast Way: 

£03) 315 

104 nn=LIWE(n,0,1) ;form array 1,...,n 

105 one=LINE(n,1,0) ;form array 1,...,1 

106 xsum=D0T(nn,one,n) ;find inner product of nn and one 
107 sum:=xsum ;convert xsum to integer 
108 LIST (sum) 

109 PAUSE 

LLON 

111 ;end 

oh 

ai Vis ks 

Eta 3 This is the End of the EXAMPLE macro 
115 ;; 


Commenting TIMESLAB Code 


As in any programming language, it is important to put comments into a 
macro file. There are two kinds of comments that can be used: (1) nondisplayed 
comments and (2) displayed comments. The first begin with a single semicolon 
and the second with two semicolons. A nondisplayed comment is ignored by 
TIMESLAB, while a displayed comment is not executed but is displayed on 
the screen (unless the PROMPTOFF command has been issued). Note that a 
nondisplayed comment line can be used to provide a target for the IF and 
GOTO commands (see below). If a semicolon appears in a line of code, then it 
and anything to its right on the line are ignored by TIMESLAB. 

A displayed comment is any line that begins with two semicolons. Such a 
line can be used to document a macro, but also has another important function. 
When TIMESLAB encounters a displayed comment line in a macro, it displays 
it on the screen (unless the PROMPTOFF command has been issued). This allows 
a macro to tell the user what it is doing as it is being executed. Note that 
almost every macro in this book begins with a prolog consisting of displayed 
comments that tell the user what the macro does and what input is required. 
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Multiple Entry Points 


A user can begin execution of a macro at any line in the file that begins 
with a semicolon and then has a string of characters by issuing the command 


MACRO(fname, string) 


where fname is the name of the file containing the macro, and string is the 
string of characters on the line where the macro is to start. In the EXAMPLE 
macro, we have provided several entry points, including the lines marked start, 
methodi, method2, method3, and method4. 


Interrupting a Macro and Providing Input 


While a macro is being executed, the user can interrupt it by striking the 
F9 function key. This causes TIMESLAB to stop executing commands from 
the macro file and to start accepting commands from the keyboard again. To 
restart the most recently interrupted macro, one need only issue the command 
MACRO with no argument. Thus for example, one can start a simulation study, 
interrupt it and look at the results so far, and then restart it. 

The ability to interrupt a macro can also be used to allow it to ask the 
user for input. The TIMESLAB command PAUSE causes TIMESLAB to pause 
in its execution of a macro until either F9 is struck (in which case the macro is 
broken and TIMESLAB accepts commands from the keyboard) or any other 
key is struck (in which case the macro resumes execution). Thus a macro can 
include some displayed comments telling the user to define some quantities and 
then a PAUSE command. When the macro is executing the user can break the 
macro, define the requested variables, and then restart the macro. Note how 
this idea was used at the beginning of the EXAMPLE macro. 


Nesting Macros 


One macro can call another macro. However, there can only be a total of 
five macros whose execution has not finished at any one time. Thus recursion 
(that is, a macro calling itself either directly or indirectly) is not allowed. Note 
that TIMESLAB has no provision for the passing of arguments when one macro 
calls another; for example, if a macro operates on an array called X, then the 
calling macro must have defined X. 


Error Handling During Macros 


When TIMESLAB starts, it is in the aborton mode; that is, an error in 
a macro causes the execution of the macro to stop at the line where the error 
occurred. If the error is easily corrected, for example, if a variable needed in the 
macro was not defined, then the user can correct it and restart the macro at the 
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line where the error occurred by issuing the command MACRO. If, however, it is 
necessary to modify the macro file in order to fix the error, then the command 
MACRO(fname), where fname is the corrected version, must be given even if the 
new file has the same name as the old file. There is a command called ABORTOFF 
which tells TIMESLAB to continue executing the macro even when confronted 
with an error. The command ABORTON cancels the effect of ABORTOFF. 


Other Basic Features 
Other basic features of macros include: 


1. When TIMESLAB starts, each line that is executed in a macro is displayed 
on the screen. This can be time consuming in a long macro. The command 
PROMPTOFF is provided to tell TIMESLAB to stop the displaying. The 
effect of this command is cancelled by the command PROMPTON. 


2. Issuing the command SPEAKERON will result in having the speaker beep 
as each line in a macro is executed. The effect of this is cancelled by the 
command SPEAKEROFF. 


3. Occasionally it is useful to execute a macro so that TIMESLAB will wait 
for a key to be struck before each command is executed. The command 
SINGLEON is provided to put TIMESLAB into this “single step” mode. 
The effect of SINGLEON is cancelled by the command SINGLEOFF. 


B.7.2. Macro Branching Commands 


There are five kinds of branching constructs available for use in macros. 
These constructs allow TIMESLAB to be used as a programming language. 
Each is illustrated in the EXAMPLE macro. 


IF(var,no,zo,po) 


This command compares the (real or integer) variable or immediate value var 
to zero and passes transfer to the line having offset no,zo, or po from the 
current line depending on whether var is negative, zero, or positive. Lines 
below the current line have positive offsets while those above it have negative 
offsets. In determining these offsets comment lines are not counted. This 
construct operates very quickly. 


IF (exp, label) 
In this command, the argument exp is one of var1.1t.var2, vari.le.var2, 


vari.eq.var2, vari.ne.var2, vari.ge.var2, or vari.gt.var2, where vari 
and var2 are again real or integer variables or immediate values. If the ex- 
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pression is true then the next line to be executed is determined by the second 
argument as follows. TIMESLAB rewinds the MACRO file and looks for a line 
that starts with a single semicolon in column 1 followed by the same character 
string as is used as the second argument. If there is no such line, a message is 
displayed and the macro aborts. 


IF(exp)...ENDIF 


The one-argument form of the IF command, together with the ENDIF command, 
affords a traditional IF...ENDIF construct. The argument of the IF command 
is the same as the two-argument form. If the expression is true, then control 
passes to the next line of the macro. If it is false, control passes to the line 
following an ENDIF that matches the IF command (such IF. ..ENDIF forms can 
be nested). 


GOTO(label) 


This command provides an unconditional branch to the line determined by the 
argument in the same way as in the two-argument IF command above. Thus 
one can use implied loops. 


WHILE(exp)...END 


This pair of commands provides a while-end construct. Unfortunately this 
construct is not particularly fast, especially when the END is far from the WHILE. 


B.8. Examples of Writing Macros 


In this section we consider several examples of writing general purpose 
macros. 


Example B.1 | FORMING TABLES 


There is no command in TIMESLAB that is specifically designed to list a 
table on the screen. There are several ways that tables can be formed however. 
The simplest case is when we have several arrays that are all of the same length 
and we would like to list them in a table so that the first array forms the first 
column, the second forms the second column, and so on. The commands 


x1=LINE(10,0,1) 
x2=x1*x1 
x=<x1,x2> 
LISTM(x,10,2) 


will form a table having two columns with the first being the numbers 1 through 
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10, and the second being the squares of the first. The LISTM command does not 
allow the user to specify the format of each row of the matrix being printed, 
and if there are more than six columns, those past the sixth in each row will 
wrap-around on the screen. In order to specify a format for a table, the LIST 
command must be used. The general form is 


LIST(x,n,m, form) 


where x is an array of length at least n whose first n elements are to be displayed 
on the screen with the first m in the first row, the next min the next row, and 
so on, and the format of each row is determined by the argument form. Thus 
for the x of the example above, if we used 


LIST. (x, 2072,2f10-4) 


we would not get what we want (the first row would be the first two elements 
which are 1 and 2, the next row would be 3 and 4, and so on). However, if 
we think of x as a (10x2) matrix, we can obtain an array for which the above 
command would work if we just transpose x first. Thus we could use 


y=TRANS(x,10,2) 
LIST(y,20,2,2£10.4) 


In the rest of this example we consider a more complicated example of 
forming a table. The DIST command will calculate values of the pdf, cdf, or 
quantile function for the standard normal, Student’s-t, y?, or F distribution. 
The macro NO1TAB calculates the cdf for the standard normal for values of z 
between 0 and 2.99 in steps of .01. Then the ability of TIMESLAB to do 
concatenation is used to create a table of these values that has the values of z 
across the top and down the left side. Note that when the LIST command is 
used, the consecutive values of an array are listed across the rows of the list. 
Thus we need to create an array that starts with a zero, then the first ten 
cdf values, then the number .1 followed by the second ten cdf values, and so 
on. This is done by thinking of the cdf array as a matrix and using the TRANS 
command and the concatenation command (the type 4 command). Once an 
array containing the rows of the table is formed, we insert at its beginning the 
values that go across the top of the table. The resulting table is displayed in 
Table B.1. This table was actually created by using the RECORD command to 
place the table created by the macro onto a file which was then modified using 
the EDIT command. The resulting file was then inserted into the file which 
printed this book. 


SEC B.8 EXAMPLES OF WRITING MACROS 455 


desis 

2a. 8 WO1TAB.MAC: macro to calculate H(0,1) cdf at 

3 4S5 z=0,.01,...,2.99 and write out table. 

Ses 

OF ace The RECORD command can be used to put this 
6: table out to a file which can be edited to 
as (ae get a nice table. 

8 55 

9 35 

10 PAUSE 

11 ;start 

12 x=LIWE(300,0,2.99,1) ;Find values at which to find cdf 
13 z=DIST(z,2,300,x) ;Calculate cdf 

14 z=TRANS(z,10,30) ;Now append left margin of table 
15 z1=LIWE(30,-.1,.1) 

16 z=<z1,z> 

17 zz=TRANS(z,30,11) 


18 z1i=LIWE(10,-.01,.01) 

19 z1i=<0,z1> 

20 zz=<z1,zz> 

21 form=’f4.1,11f6.4’ ;Format for each row of the table 
22 LIST(zz,341,11,form) ;Write out table 


Example B.2 | A SIMULATION STUDY 


In this example we construct Figure B.2 where we have superimposed the 
density estimators of a set of 1000 sample means and medians for N (0,1) white 
noise series of length 10. Note how both graphs appear normally distributed 
and that the one for the sample means is more highly concentrated. This 
is interesting because asymptotic theory says that the limiting ratio of the 
variance of the sample mean to the variance of the sample median is 2/7 as 
n — oo, which we are seeing here (the ratio of the values of normal pdf’s at 
zero is the ratio of their standard deviations) for a sample size as small as 10. 
Note that Figure B.2 was produced by MEDMN.MAC with nsamps=1000, n=10, 
dist=1, npts=50, rbins=4., and kernel=3. Since it is difficult to predict 
what a good scale would be on the axes in PLOTK, we waited to see the result 
of the macro and then used 


PLOTK(y,fy,npts,2,type,-1.5,1.5,0,1.2) 


to get the nice tic mark labels in Figure B.2, and then used the FIND utility 
(see Section B.5.5) to put the labels on the graph before getting a hard copy 
of the screen by striking F3 from the graphics menu. 

Note that this macro can be thought of as a model for almost any simu- 
lation study. It takes a long time to generate the 1000 samples since we are 
using TIMESLAB as an interpretive language. Once we have the arrays xmn 
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Table B.1. Table of Standard Normal Cumulative Probabilities 
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and xmed, we can modify npts, rbins, and kernel and recall the macro by 
the command 


MACRO(MEDMN , density) 


which will go directly to the comment labeled ;density, and thus avoid recal- 
culating the means and medians. 

This macro illustrates several other things as well. First, the LINE com- 
mand is used in two different ways: (1) to form arrays xmed and xmn, each of 
length nsamps which will receive the means and medians for each sample, and 
(2) to form an array of length n which is used later to find the sample mean as 
an inner product of the sample and the array of ones divided by n. Next, the 
PROMPTOFF command is used at the beginning since echoing the thousands of 
commands to the screen will cause the macro to take much longer. After each 
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Density for Mean and Median (=1@) 
The wean is the selid curve 


ath 


0.0 
“158-98 38 Fe 1.50 


Figure B.2. Using DENSITY to Compare the Sampling Distribution of the 
Sample Mean and Median for Samples of Size 10 from a N(0,1) Population. 


mean and median is calculated, their value and which sample they correspond 
to are listed via the LIST command. Otherwise, the user would have no idea 
that anything was happening while the macro is executing. Third, note that 
the first time WN is called the user-supplied seed is used while henceforth using 
seed=0 causes WN to use the output seed of the last time WN was called. 

Several times in the macro, individual elements of arrays are referred to by 
using the square brackets [...], while the ability to concatenate arrays using 
<...> is used a few times. Note that we found the median of a data set by 
sorting it and then finding the middle value (or the average of the two middle 
values if n is even). Finally, note the use of the implied loop using the IF and 
GOTO commands. 


1 553 

ane MEDMN.MAC: macro to generate nsamps white noise series 

iP RS of length n (using seed as random number 

= ety: generator seed) having distribution specified by 
Sas the integer dist and find the sample mean and 
Ope median for each one. Then the density command is 
| ae used to estimate the pdf for the means and medians. 
8 33 

ait INPUT: nsamps,n,seed,dist, rbins,npts,kernel 

LO 3s 

11 PAUSE 

12 ;start 

LS ys 

14 PROMPTOFF ;don’t waste time showing commands 

15 xmn=LINE(nsamps,0,0) ;Teserve space for means 


16 xmed=LINE(nsamps ,0,0) jand medians 
17 one=LINE(n,1,0) . ;we’ll use this array in finding means 
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18 ni={nti}/2 ;the median will be the average 

19 n2={n+2}/2 ;of the sorted x’s with these indices 
20 on:=n 

21 ns=1 

22 x=WN(seed,n,dist) ;warm up random number generator 

23 ; 

24 ;startloop 

2a. 

26 x=WH(0,n,dist) ;generate sample 

27 x=SORT(x,n) ;sort data 

28 xmed[ns]={x[n1]+x[n2]}/2 ;find median 

29 c=DOT(one,x,n) ;find mean 

30 xmn[ns]=c/on 

31 LIST(ns,xmn[ns] ,xmed[ns]) ;list sample number and mean and median 
32 IF(ns.eq.nsamps,density) ;finished all samples? 

33 ns=nst+1 ;no 

34 GOTO(startloop) 

35 ; 

36 ;density ;yes 

ST 


38 DENSITY (xmed,nsamps ,rbins ,npts, kernel , ymed, fymed) 

39 DENSITY (xmn ,nsamps ,rbins,npts ,kernel, ymn,fymn) 

40 y=<ymed, ymn> ;prepare to use PLOTK 

41 fy=<fymed,fymn> 

42 LABEL(y)=? ? 

43 LABEL(fy)=’Density for Mean and Median (n=#n#)? 

44 type=<2,33> 

45 PLOTK(y,fy,npts,2,type) 

46 PROMPTON ;turn prompt back on 


Example B.3 | ARRANGING CALCULATIONS TO SAVE TIME 


Every time a command is received by TIMESLAB, it must be interpreted, 
a process that takes a small amount of time. In fact, for a command that does 
not perform a large number of calculations, more time is spent interpreting 
the command than doing the calculations. This is particularly important in 
macros that have looping constructs, since an IF command whose condition is 
true causes TIMESLAB to rewind the file and then start reading through it 
looking for its target line. 

To illustrate this, consider a macro that will generate 50 samples of size 10 
from a uniform population and find the sample mean for each. The following 
macro does this using both a loop and a much faster way. The slow way takes 
15 seconds on a standard PC/AT, while the fast way takes 0.5 second. This 
is an extreme example, but it is important to try to organize calculations as 


much as possible to reduce the amount of time required to perform a given 
task. 
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omMN Oak WN eH 


rR 
- Oo 


ne SLOW.MAC: This macro illustrates how some sequences of 

33 commands can be incorporated into a single command. 
A We’1l generate 50 samples of 10 uniforms each 

sae in two ways: (1) in a loop (very slow), and 

33 (2) in a single command (very fast). 


any Input : none 

>> 

PAUSE 

;start 

CLS ;clear screen 


sis Slow Way (this takes a while, I’1l keep you informed 
35 as to which sample we’re doing) 
means=LIWE(50,0,0) 

ns=1 

seed=12345. 

x=wn(seed,10) 

TIME(t1i) 

PROMPTOFF 

;startloop 

x=WH(0,10,2) 

x=SUBMNS (x ,10,1,means[ns] ,0) 

LIST(ns) 

IF (ns .eq.50,endloop) 

ns=nsti 

GOTO(startloop) 


;endloop 


TIME(t2) 
time=t2-t1 
LIST(time) 

PAUSE 

HIST (means ,50,5) 
CLS 

PROMPTON 


Ae FAST WAY: 


;fast 

seed=12345. 

x=wn(seed,10) 

TIME(t1i) 

x=WH(0,500,2) 
x=TRAHS (x, 10,50) 
x=SUBMHS (x ,500 ,50 ,means ,0) 
TIME(t2) 

time=t2-t1 

LIST (time) 
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PAUSE 
HIST (means , 50,5) 


Example B.4 | USING THE SPEAKER 


The SPEAKER command can be useful in a macro to alert the user that a 


macro has ended or else that a certain part of the macro has finished. Just for 


fun, 
Joy” 


omMmnN aonb wonder 


NNN NFP PRP RPP RP BP RB Re 
WBnNHroewmNt anh WNRr OO 


we include the following macro that will produce the theme from “Ode to 
on the speaker when invoked. 


He ODE.MAC: macro to play ‘Ode to Joy’ on the speaker. 
ais To speed it up or slow it down, just 
nie decrease or increase the value of nh. 


La 

PAUSE 

;start 

nh=30 

nh2=2*nh 

£=<659 ,659 , 698 ,784, 784 ,698 ,659 ,587 ,523 ,523> 

f=<f ,587 ,659, 659 , 587 ,587,659 ,659 ,698 ,784,784> 

f=<f ,698 ,659 ,587 ,523 ,523, 587 ,659 ,587 ,523,523> 

f=<f ,587 ,587 ,659 ,523 ,587 ,698 ,659 ,523,587 ,698> 
=<f ,659 ,587 ,523 ,587 ,784,659 ,659 ,698,784,784> 

f=<f ,698 ,659, 587 ,523 ,523,587 ,659 ,587 ,523 ,523> 
t=<nh ,nh,nh,nh,nh,nh,nh,nh,nh,nh> 

t=<t ,nh,nh,nh,nh,nh2,nh,nh,nh,nh,nh> 

t=<t ,nh,nh,nh,nh,nh,nh,nh,nh,nh,nh2> 

t=<t ,nh,nh,nh,nh,nh,nh,nh,nh,nh,nh> 

t=<t ,nh,nh,nh,nh,nh2,nh,nh,nh,nh,nh> 

t=<t ,nh,nh,nh,nh,nh,nh,nh,nh,nh,nh2> 
SPEAKER(f ,t ,60) 


Another interesting use of the speaker is to “play an array,” that is, to take 


the successive elements of an array and have the speaker sound in a frequency 
proportional to the magnitudes of the elements. In the PLAY macro, we form 
the plot of an array and then while the plot is on the screen, we play it on the 
speaker. The user must specify the smallest and largest frequencies wanted. 


eonoanrrhwnre 


oF PLAY.MAC: macro to ‘play an array,’ i.e. to take the first 
Be n elements of the array x and make the speaker 
a sound at a frequency proportional to each element. 


aa INPUT: x, n (array to be played and how many elements to play) 
oie fmin, fmax (minimum and maximum frequencies to use; 
Ae these should be between 25 and 20,000) 
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OM EA time (number of hundredths of seconds each tone is 

10 ;; to play; this should be between 1 and 100) 

11 ;; 

12 PAUSE 

13 ;start 

14 PLOTON }g0 to graphics mode 

15 BATCHON ;no pause after PLOT 

16 PLOT(x,n) ;plot data 

17 MAXMIN(x,n,xmax,im,xmin,im) ;convert range of data to [fmin,fmax] 


18 f£=fmint{{x-xmin}/{xmax-xmin}}*{fmax-fmin} 
19 t=LIWE(n,time,0) ;form array of times 


20 SPEAKER(f,t,n) ;play array 
21 BATCHOFF jturn batch mode off 
22 PLOTOFF ;g0 back to text mode 


Example B.5 | DRAWING A CIRCLE 


Interesting effects can be created on a graphics screen by using the PLOT2 
command with the option to connect the two arrays being plotted. For exam- 
ple, the following macro will draw a solid circle on the screen. One interesting 
aspect of this is the fact that in order to get the circle to appear actually round, 
we had to experiment with the scaling factors in the PLOT2 command as the 
pixels on the screen are not square. In fact, on most monitors, the ratio of the 
height to width of a pixel is approximately 12/5. 


PSS 

b sen ks CIRCLE.MAC: macro to draw a circle on a graphics screen. 
3 53 

4 ;; INPUT: none 

5 55 

6. 3s OUTPUT: none 

Oars 

8 PAUSE 

9 j;start 

10 x=LIWE(480,-1,1,1) ;x is -1,...,1 (use 480 so every column of 
es ;pixels gets turned on) 

12 yi={i-x*x}~.5 ;y1 is top half of circle 

13 y2=-y1 ;y2 is reflection of yi 

14 LABEL(y1i)=’ ° 

15 LABEL(y2)=" ? 

16 LABEL(x)=’ ” 

Riot 

1a: 5 Plot on a scale so that the circle looks round. 

i s 


no 
o 


PLOT2(y1 ,y2,480,480,3,3,x,-1.75,1.75,-1,1) 
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Example B.6 | SUPERIMPOSING PLOTS 


An important use of graphics in TIMESLAB is to superimpose many plots 
on the same screen. In this example we describe three macros that use this 
ability. 

The HAT2 Macro 


The HAT2 macro was used to form the top graph in Figure B.3. The macro 
calculates the (n x n) hat matrix 


HS X(X7 x) x 


for the (n x 3), equally spaced quadratic regression matrix 


Ise} AL 

i 7 ah 

IeicineO 
x= 

ton) a 


Then a graph of each of the first [n/2]+1 rows (versus their column numbers) 
is produced on the screen. The only input required from the user is the value 
of n. In Figure B.3 we used n = 40. 

The macro also illustrates how labels can be added to a screen using the 
LABEL command. 


Lis.is 

2 ;;  HAT2.MAC: macro to plot the first [n/2]+1 rows of the 
oh oe (n x 3) quadratic regression matrix 

4. GK 

OP rai X = <1,x,x72>, where x=<1,2,...,m 

GS, 

1h Fe INPUT: n (less than or equal to 80) 

8 35 

9 PAUSE 

10 ;start 

11 PLOTON ;Switch to graphics mode 

12 BATCHON ;don’t want to pause between graphs 
13 one=LINE(n,1,0) ;first column 

14 X=<one> 

15 x=LIWE(n,0,1) ;second column 
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ist 21 rows of W for Quadratic Reg 


16.@ 24.8 
Element of Row 


Chi-Square pdf for df=10,20,...,208 


S¢ SSR 
DOO 


"¢.0 50.08 100.0 150.0 20.08 250.08 


Figure B.3. Three Examples of Superimposing Plots on a Graphics Screen. 
The Top Graph Is the Output of the HAT2 Macro; the Middle and Bottom 
Graphs Are from the CHIPLOT and FPLOT Macros Respectively. 
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16 X=<X,x> 

17 x=x"2 ;third column 

18 X=<X,x> 

19 m=3 

20 XTX=MMULT(X,X,n,m) 3X transpose X 

21 XTX=MINV(XTX,m,ier) ;inverse of X transpose X 


22 hi=MMULT(X,XTX,n,m,m) 
23 XTX=TRANS(X,n,m) 


24 HAT=MMULT(h1,XTX,n,m,n) ;HAT is the hat matrix 

25 n2=n7"2 

26 MAXMIN(HAT ,n2,hmax,im,hmin, im) ;need max and min to scale plots 
27 i=1 ;now we’1l loop over rows of HAT 

28 nb2={n/2}+1 ;only need to do first ‘half’ since symmetric 
29 5 

30 ;startloop ;this is target of implied loop 

31 ; 

32 i1={i-1}«*n+1 ;i1 and i2 are begin and end of ith row 

33 i2=i+*n 


34 y=EXTRACT(HAT,i1,i2) ;pull off ith row 
35 LABEL(y)=’Row #i# of H for Quadratic Reg’ 


36 PLOT(y,n,0,n,hmin,hmax) ;plot ith row 

37 LABEL(150,140,1)=” (n=#n#) ? ;put value of n on plot 

38 if(i.eq.nb2,endloop) ;check if we’ve done all rows 
39 i=i+1 ;no 

40 GOTO(startloop) 

a1 Gs 

42 ;endloop ;yes 

43 ; 

44 LABEL(y,0)=’Hat Value’ ;put label on vertical axis 


45 LABEL(x,0)=’Element of Row’ ;put label on horizontal axis 
46 LABEL(cap,0)=’ist #nb2# rows of H for Quadratic Reg’ 
47 GRMENU(O,n,hmin,hmax) ;put graphics menu on the screen 


The CHIPLOT Macro 
In this macro, we superimpose the graphs of the pdf of the x? distribution: 


1 


% —2/2,(k/2)-1 
PE (k/D* & ; z>10, 


f(z) 
where I is the gamma function 
co 
(a) = / ew id? a> 0, 
0 


for degrees of freedom k given by k = 10,15, 20,..., 200. Each graph is plotted 
only from the fifth to the ninety-fifth percentile of the distribution. 
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LG 

2 ;; CHIPLOT.MAC: macro to superimpose the graphs of the 
a” =s chi square pdf for degrees of freedom 
433 10 through 200 in steps of 10. Each 

Sy oe plot is from the 5th to the 95th percentile. 
6 55 

uf os INPUT: none 

8 33 

9 PAUSE 

10 ;start 

11 PLOTON ;switch to graphics mode 

12 BATCHON ;nho pause between plots 

13 n=10 

14 ; 

15 ;startloop 

HGy 38 

17 c1i=DIST(c,3,1,.05,n) ;find 5th and 95th percentile 
18 c2=DIST(c,3,1,.95,n) 

19 x=LIWE(100,ci,c2,1) ;generate values at which to find pdf 
20 c=DIST(c,1,100,x,n) ;find pdf 

21 LABEL(x)=? ” 

22 LABEL(c)=’Chi Square pdf, df=#n#,#n#? 

23 PLOT(x,c,100,0,250,0, .10) splot it 

24 IF(n.eq.200,end) jall done? 

25 n=n+i0 ;no 

26 GOTO(startloop) 

ati 

28 ;end ;yes 

20 

30 LABEL(cap,0)=’Chi-Square pdf for df=10,20,...,200” 

31 GRMENU(0,250,0, .10) ;let user FIND or PRINT,etc 
32 BATCHOFF ;return to normal 

33 PLOTOFF 


The FPLOT Macro 


465 


In this macro, we superimpose the graphs of the pdf of the Fy, %, distri- 


bution: 


ki/2 
pi 0e1/2)=2 
ko 


Sy k (ki +k2)/2’ 
B( 4,2) (14+ 
oe, ko 


where £@ is the beta function 


f(z) = x (Up 


T'(a + b) 


Aa, 6) = Taree) 


for degrees of freedom k; and ky given by n = ky = kp = 10,15,20,..., 200. 
Each graph is plotted only from the fifth to the ninety-fifth percentile of the 


distribution. 
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eve FPLOT.MAC: macro to superimpose the plots of the F 

ahs pdf for degrees of freedom (n,n) for n 

OF between 10 and 200 in steps of 5. The plots 
she are drawn from the 5th to 95th percentiles. 


Sif INPUT: none 

> 

PAUSE 

;start 

PLOTOE 

BATCHOE 

n=10 

;startloop 
£1=DIST(£,3,1,.05,n,n) 
£2=DIST(£,3,1,.95,n,n) 
x=LIWE(100,f1,£2,1) 
f=DIST(f,1,100,x,n,n) 
LABEL(x)=? ” 


;switch to graphics mode 
;no pause between plots 


;find 5th and 95th percentiles 


;generate values at which to find f 
;generate values of the pdf 


LABEL(f)=’F pdf, df=#n#, #n#’ 


PLOT(x,f,100,0,3,0,3) 
IF (n.eq.200,end) 
n=n+5 
GOTO(startloop) 


, 


send 


;plot pdf 
;all done? 
;no 


;yes 


LABEL(cap,0)=’F pdfs for df=10,15,...,200” 


GRMEHU(0,3,0,3) 
BATCHOFF 
PLOTOFF 


;put up graphics menu 
;return things to normal 


APPENDIX C 


The TIMESLAB Commands 


This appendix has two sections. In the first we give an outline of the 
TIMESLAB commands according to their function. In Section C.2 we describe 
each command in detail. 


C.1. Outline of Commands 


The TIMESLAB commands can be loosely grouped into the following 
categories by their purpose: 


COLORS ESTIMATION MATRICES SIMULATION 
CORRELATIONS GRAPHICS MODELS SPEAKER 
DISTRIBUTION I0 POLYNOMIALS SPECTRA 
DOS LISTING PREDICTION TRANSFORMS 
EDITOR MACROS PRINTING WNTESTS 


ENVIRONMENT MATH 


COLORS 


The colors in the text and graphics modes can be modified via the TEXT- 
COLOR and COLOR commands respectively. The colors that are available are 
listed under these commands in Section C.2. 
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CORRELATIONS 
TIMESLAB has the following commands for manipulating auto- and cross- 
correlations: 
ARCORR Find correlations corresponding to AR parameters. 
ARCORR2 Find correlations corresponding to bivariate AR parame- 
ters. 
ARMACORR Find correlations corresponding to ARMA parameters. 
ARPART Find partial correlations corresponding to AR parameters. 
CORR Find sample correlations and/or spectral density for a 
data set. 
CORR2 Find sample auto- and cross-correlations for bivariate 
data. 
CORRAR Find AR parameters from correlations. 
CORRAR2 Find AR parameters from bivariate correlations. 
CORRARMA Find ARMA parameters from correlations. 
CORRMA Find MA parameters from correlations. 
MACORR Find correlations corresponding to MA parameters. 
PARCORR Find partial correlations and standardized residual vari- 
ances from data. 
PARTAR Find partial correlations from AR parameters. 
DISTRIBUTION 


TIMESLAB has the following commands for studying the distribution of 


data: 
DENSITY 
DIST 
HIST 
INFQNT 


DOS 


Nonparametric probability density estimation. 
Pdf, cdf, or quantile function of Z, t, x”, or F distribution. 
Calculate and plot histogram. 


Calculate and plot informative quantile function. 


It is possible to issue DOS commands from within TIMESLAB by using 
the command called DOS. 
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EDITOR 


TIMESLAB has its own built-in text editor (called EDIT) for creating and 
modifying data or MACRO files. 


ENVIRONMENT 

The environment under which TIMESLAB operates can be varied by using 
the following commands: 

BATCHOFF - Cancel BATCHON. 


BATCHON - Henceforth there will be no pause after a plot is produced 
(this can be used in conjunction with PSON to produce a 
printed copy of an unattended session). 


CLEAN - Delete a variable or variables. 

CLS - Clear screen. 

COLOR - Set colors in the graphics mode. 

ECHO - Same as Ctrl-Prtsc, that is, toggles printer echo. 
OVEROFF - Henceforth any command that would cause the replace- 


ment of an existing variable will cause TIMESLAB to ask 
permission before doing the replacement. 


OVERON - Any command that would replace an existing variable will 
do so. 

PLOTOFF - Switch to the text mode. 

PLOTON - Switch to the graphics mode. 

PLOTSIZE ~- Set graphics parameters (size and location of plots, num- 


ber of tic marks, and number of characters in tic mark 
numerical labels). 


PRINTER - Issue commands to the printer. 

PRINTSEL - Tell TIMESLAB what kind of printer is being used. 
PSOFF - Cancel PSON. 

PSON - Henceforth any plot on screen will also go to the printer. 
RECORD - Keep a record of the session on a file. 

RESTART - Equivalent to QUIT and then restarting TIMESLAB. 


TEXTCOLOR - Set colors in the text mode. 
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TIMESLAB has the following commands for estimation: 


ARMASEL 
ARSP 
ARSP2 


ARSPCB 


ARSPPEAK 
COEFFCSD 


CORR 
CORR2 
CORRAR 
CORRAR2 
CROSSP 
DTAR 
DTARMA 
SEASEST 
WINDOW 


GRAPHICS 


Subset univariate ARMA fitting and estimation. 
Used with DTAR for AR spectral estimation. 


Use with CORR2 and CORRAR2 to do bivariate AR spectral 
estimation. 


Used to find simultaneous confidence bands for AR spec- 
tra. 


Used to find estimates of peak frequencies for AR spectra. 


Find asymptotic standard errors for ARMA parameter 
estimates. 


Univariate sample correlations and spectral density. 
Auto- and cross-correlations for bivariate data. 

Use with CORR to do univariate AR modeling. 

Use with CORR2 to do bivariate AR modeling. 
Bivariate nonparametric spectral estimation. 
Univariate AR parameter estimation. 

Exact MLE’s for ARMA parameters. 

Approximate MLE’s for Box-Jenkins type models. 


Univariate nonparametric spectral estimation. 


TIMESLAB has the following high resolution graphics commands: 


CLS 
ERASE 
FIND 


GRMENU 
HIST 
INFQNT 


Erase a graphics screen. 
Erase part of a graphics screen. 


TIMESLAB has an interactive locator/labeler utility for 
graphics screens called FIND. For information about FIND, 
see Section B.5.5. 


Place the graphics menu on a graphics screen. 
Calculate and plot histogram for an array. 


Calculate and plot informative quantile plot for an array. 
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LABEL 


PLOT 
PLOT2 


PLOTCSP 


PLOTK 
PLOTOFF 
PLOTON 


PLOTSP 


RESCREEN 


SAVESC 


IO 
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Define a new plotting label for an array to be plotted 
or place a character string onto an existing screen. For 
information about labeling plots, see Section B.5.7. 


General plotting of Y versus X. 
Superimpose 2 Y versus X plots. 


Plot a spectral array with vertical axis not on log scale, 
horizontal axis from 0 to .5 cycle per unit time. 


Superimpose K Y versus X plots. 
Switch to the text mode. 
Switch to the graphics mode. 


Plot a spectral array with vertical axis on log scale, hori- 
zontal axis from 0 to .5 cycle per unit time. 


Retrieve previously saved graphics screen or screens and 
put back on screen. 


Save a compressed image of a graphics screen to a file. 


The TIMESLAB commands for reading data from a disk file and writing 
data to a disk file from within TIMESLAB are called READ and SAVE respec- 


tively. 


LISTING 


TIMESLAB has the following commands for producing textual displays 


on the screen: 


INFO 


LIST 


LISTM 


LISTSP 


Display information about all quantities that have been 
defined. 


Display a set of scalar variables or the elements of an 
array. 


Display the elements of a two- or three-dimensional array. 


Display the elements of a spectral array and the corre- 
sponding frequencies. 
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TIMESLAB can be used as an interpreted programming language by form- 
ing a disk file (via the EDIT command) containing a series of TIMESLAB com- 
mands, and then telling TIMESLAB to execute the commands in the file by 
using the MACRO command. Such a file is referred to as a macro. 


The execution of a macro can be interrupted by striking the F9 function 
key. It can be restarted by issuing the MACRO command with no argument. 
While the macro has been interrupted, other commands can be issued from 
the keyboard. For more information and examples, see Section B.8. 


The following commands are used with macros: 


ABORTOFF - 


ABORTON = 


DELAY = 


PAUSE = 


PROMPTOFF - 


PROMPTON - 


SINGLEOFF - 


SINGLEON” - 


SPEAKEROFF - 


SPEAKERON- - 


TIME - 


Henceforth anytime a macro ecounters an error, it will 
continue executing the commands in the macro. 


Henceforth anytime a macro encounters an error it will 
stop. 


The macro pauses for a specified amount of time before 
resuming execution. 


When this command is encountered, TIMESLAB issues a 
message to the screen and awaits the user’s response. If 
the user precedes this command with comments that start 
with two semicolons (which causes them to be displayed 
on the screen), the user can break the macro, provide 
information that the macro needs, and then restart the 
macro. 


Henceforth the commands being executed will not be dis- 
played on the screen. 


Henceforth each command being executed will be dis- 
played on the screen. 


Cancel SINGLEON. 


Henceforth the macro waits for the user to strike a key 
before executing each command. 


Henceforth the speaker will not sound as each command 
is executed. 


Henceforth the speaker will sound as each command is 
executed. 


Determines elapsed time since midnight. TIME can be used 
to time how long a series of commands takes to execute. 
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MATH 
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TIMESLAB has the following commands for doing mathematical opera- 


tions: 


TYPE2 


ABS 
ARDT 
ARDT2 
BINOM 
cos 
CUM 
DIST 
EXP 
LINE 
LOGE 
POLY 
SIN 
SORT 


This is actually a set of commands for doing arithmetic 
operations such as adding, subtracting, multiplying, di- 
viding, and exponentiating reals, integers, and arrays as 
well as simple assignment statements. 


Absolute value function. 

Generate values of a difference equation. 
Generate values of a bivariate difference equation. 
Find binomial coefficients or probabilities. 
Cosine function. 

Cumulative sums or averages of an array. 
Pdf, cdf, quantile of Z, t, y?, F distributions. 
Exponential function. 

Form a line. 

Natural log function. 

Evaluate a polynomial. 

Sine function. 


Sort an array. 


For information on what vector and matrix commands are available, see 
the MATRICES section that follows. 


MATRICES 


TIMESLAB has the following commands for doing vector and matrix op- 


erations. 
DOUBLE 
EIG 
EXTRACT 
GS 
LISTM 
MCHOL 


Extend an array by symmetry. 

Find eigenvalues and (optionally) eigenvectors. 

Use to extract parts of arrays. 

Gram-Schmidt decomposition. 

List the elements of a matrix or three-dimensional array. 


Modified Cholesky decomposition. 
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MDEL - Delete some rows of a matrix. 
MINV - Invert a matrix. 
MMULT - Multiply matrices. 
REVERSE - Form an array by reversing the order of the elements of 
another array. 
SWEEP - Sweep a matrix. 
TOEPL - Form a Toeplitz matrix. 
TRANS - ‘Transpose a matrix. 
TYPE4 - Use to concatenate matrices. 
MODELS 


The TIMESLAB commands for studying models fall into the following 
areas: 


CORRELATIONS ESTIMATION PREDICTION SIMULATION 
SPECTRA 


For the commands in each of these areas, see the individual area descrip- 
tions in this section. 


POLYNOMIALS 


TIMESLAB has the following commands for studying polynomials: 


INVPOLY - Find the coefficients of the reciprocal of a polynomial. 
MULTPOLY ~- Find the coefficients of the product of two polynomials. 
POLY - Evaluate a polynomial. 


POLYROOTS - Find the roots of a polynomial given its coefficients. 
ROOTSPOLY - Find the coefficients of polynomial given its roots. 


PREDICTION 


TIMESLAB has the following commands for doing prediction: 


ARMAPRED - Prediction for ARMA models. 
DTFORE - Iterated AR forecasting. 
EXTEND - Inverse operation to differencing. 


SEASPRED - Box-Jenkins forecasting. 
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PRINTING 
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TIMESLAB has the following commands for getting printed output: 


ECHO ~ 
LIST = 


PAGE - 
PRINT - 
PRINTER - 
PRINTSEL~ - 
PSON - 


RECORD = 


SIMULATION 


Same as Ctrl-Prtsc, that is, toggles printer echo. 


The display of variables can be directed to the printer via 
the LIST command. 


Position printer to top of the next page. 

Print a set of scalar variables or the values of an array. 
Send control sequences to printer. 

Tell TIMESLAB what type of printer is being used. 


Henceforth any plot on screen will go to the printer (can- 
celled by PSOFF). 


Keep a record of a session on the printer. 


TIMESLAB has the following commands for doing simulation: 


ARDT - 
ARDT2 - 
ARMA - 
MADT = 
WN = 


SPEAKER 


Simulate data from an AR process. 
Simulate data from a bivariate AR process. 
Simulate data from an ARMA process. 
Simulate data from an MA process. 


Simulate data from a white noise process. 


TIMESLAB has a command called SPEAKER which allows the user to con- 
trol the PC’s speaker. This can be useful in macros to alert the user that 
certain parts of the macro have been reached. 


SPECTRA 


TIMESLAB has the following commands for spectral analysis: 


ARMASP - 
ARSP -: 
ARSP2 - 
ARSPCB - 


Spectra for ARMA process. 
Spectra for AR process. 
Spectra for bivariate AR process. 


Simultaneous confidence bands for AR spectra. 
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ARSPPEAK 
CORR 
CROSSP 
CUMSP 
FFT 

MASP 
POLAR 


WINDOW 


TRANSFORMS 


THE TIMESLAB COMMANDS APP 2G. 


Estimate frequency of peak in AR spectra. 

Periodogram and/or correlations of univariate data. 
Nonparametric cross-spectral analysis for bivariate data. 
Cumulative spectra. 

Fast Fourier transform. 

Spectra for MA process. 

Find amplitude and phase of complex array. 


Nonparametric spectral estimation for univariate data. 


TIMESLAB has the following commands for transforming data: 


ARDT 
ARDT2 
ARFILT 
DIFF 
DIVSDS 
EXP 
FFT. 
FILT 
LOGE 
REG 
SORT 


SUBMNS 


WNTESTS 


Univariate difference equation. 
Bivariate difference equation. 
Apply AR filter. 

Find difference of data. 

Calculate and divide seasonal standard deviations. 
Exponential function. 

Fast Fourier transform. 

General linear filtering operation. 
Natural log. 

Regression analysis. 

Sort an array. 


Calculate and subtract means. 


TIMESLAB has two commands for testing for white noise: 


BARTTEST 
QTEST 


Bartlett’s test. 


Portmanteau test. 
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C.2. Detailed Description of Commands 


In this section we describe the purpose, format(s), input and output argu- 
ments (if any) for each of the commands as well as any remarks about special 
features of the command. In Appendix E of this book is a short list of all of 
the commands in TIMESLAB together with where in the book the command 
is introduced. Note that in the description of the format of a command, any 
argument that is placed in square brackets, such as in 


y=DIST (name, iopt ,n,x[,df1,df2]) 


is an optional argument, that is, sometimes it is needed and sometimes it is 
not. 


ABORTOFF Command 


Purpose: 

Cancel effect of ABORTON; that is, henceforth macros that encounter an 
error during their execution will continue running. 
Format: 


ABORTOFF 


ABORTON Command 


Purpose: 

Henceforth any error encountered during execution of a macro will cause 
the macro to stop at that line. 
Format: 


ABORTON 


Remarks: 
1. ABORTON is cancelled by ABORTOFF. 
2. ABORTON is in effect when TIMESLAB is started. 


3. If a macro is aborted, one can usually fix the error and then restart the 
macro at the line causing the error by issuing the command MACRO. 
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ABS Command 


Purpose: 


To find the absolute value of a scalar or of the first n elements of an array. 


Format: 
1. Y=ABS (X) 
2. Y=ABS(X,n) 
Input: 
x - In the first form a real scalar and in the second form an array 
of at least n elements. 
n - An integer containing the number of elements of X for which 
to find the absolute value. 
Output: 
W - In the first form a real scalar, and in the second form an array 


of n elements. 


a ae ee ee eee! ae 
ARCORR Command 
oe EN Sf en ON ee 


Purpose: 


Calculate the variance R(0) and autocorrelations p(1),...,(M) of an 
AR(p, a, 07) process. 


Format: 


rho=ARCORR(alpha,p,rvar,M,RO, ier) 


Input: 
alpha - Array of length p containing coefficients a. 
p - Integer containing order p (>0). 
rvar - Real scalar containing error variance o? (>0). 
M - Integer containing the number of autocorrelations to calculate 


(20). 


ssc C.2 


Output: 
RO 
ier 
rho 


Remarks: 
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Real scalar containing variance R(0) of process. 


Integer variable indicating whether or not the AR process is 
stationary (0 means yes, anything else means no). 


Array of length M containing autocorrelations p(1),..., o(M). 


1. If ier# 1, then RO and rho are not calculated. 


2. If M=0, then RO is calculated but rho is not. 


3. Workspace=M+p. 


Purpose: 


ARCORR2 Command 


To find the auto- and cross-correlations for a bivariate AR process. 


Format: 


ARCORR2(A,p,sigma,M,R10,R20,rho120,rho1,rho2,rho12,rho21,ier) 


Input: 
A 


R20 


rhoi20 
rhol 


Array of length 4*p containing AR coefficient matrices (the 
first two elements are the first column of first matrix, next 
two are second column, etc.). 


Integer (> 0) containing AR order. 
Array of length 4 containing error covariance matrix. 


Integer containing the number of correlations to calculate. 


Real scalar variable containing the variance of the first uni- 
variate process. 


Real scalar variable containing the variance of the second 
univariate process. 


Real scalar variable containing the cross-correlation of lag 0. 


Array of length M containing the autocorrelations of lags 1 
through M for the first univariate series. 
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rho2 - Array of length M containing the autocorrelations of lags 1 
through M for the second univariate series. 

rhoi2 - Array of length M containing the cross-correlations of lags 1 
through M. 

rho21 - Array of length M containing the cross-correlations of lags —1 
through —M. 

ier - Integer variable which is 0 if the process is judged to be stable 
(all zeros of determinental polynomial outside the unit circle) 
and 1 if not. 

Remarks: 


1. If ier=1, then the other output variables are not returned. 


2. Workspace=5 (M+1)+9p+2. 


eee EE Se ee oe Cie Tr was 
ARDT Command 


—S——————— eee 


Purpose: 


1. Generate a realization of length n from a Gaussian AR(p,a@, a7) process 
X or 


2. Generate successive values of a general difference equation: 


e(2), Salhsner 
XO. any lincspi at gh aman . 
e(i) — jai a(9)X (i — 9), t=p-+,...,7. 
Format: 
1. X=ARDT(alpha,p,rvar,seed,n,ier,RO) 


2. X=ARDT(alpha,p,n,e) 


Input: 
alpha - Array of length p containing coefficients a. 
p - Integer containing order p (>0). 
rvar - Real scalar containing error variance o? (>0). 
seed - Real scalar containing the seed for the random number gen- 


erator. 


n - Integer (>p) containing the length of the realization. 
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e - Array of length n containing the values of e for the general 
difference equation. 


Output 
ier - Integer variable indicating whether or not the AR process is 
stationary (0 means yes, anything else means no). 
RO - Real variable containing the variance R(0) of the process. 
X - Array of length n containing realization. 
Remarks: 


1. If ier=1, then RO and X are not calculated. 


2. Workspace=n+p+p? for form 1, and n+p for form 2. 


ARDT2 Command 


Purpose: 


1. Generate a realization of length n from a bivariate Gaussian AR(p, A, Z) 
process X or 


2. Generate successive values of a general difference equation: 


e(i), fal eee 


xQ)= {ene P_, A(j)X(i- 9), pp eden: 


Format: 
1. X=ARDT2(A,p,sigma,seed,n,ier) 
2. X=ARDT2(A,p,n,e,ier) 


Input: 

A - Array of length 4*p containing AR coefficient matrices (the 
first two elements are the first column of first matrix, next 
two are second column, etc.). 

Pp - Integer containing AR order. 

sigma - Array of length 4 containing error covariance matrix. 

seed - Real scalar containing the seed for the random number gen- 


erator. 
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n - Integer (>p) containing the length of the realization. 


e - Array of length 2*n containing the values of e for the general 
difference equation. 


Output 
ier - Integer variable indicating whether or not the difference equa- 
tion is stable (0 means yes, anything else means no). 
X - Array of length 2*n containing realization. 
Remarks: 


1. If ier=1, then X is not calculated. 
2. The first two elements of X are X(1), the next two are X(2), and so on. 
3. Workspace=4(n+p+1) for form 1, and 4(n+p) for form 2. 


Se ee eee ee. 
ARFILT Command 
a a a 


Purpose: 


To apply an AR(p, @, 0”) filter to an (n xm) matrix X, that is, to multiply 
X by D~1/?L~! where V = LDL? is the Cholesky decomposition of the (nxn) 
covariance matrix of the AR process. 


Format: 


Y=ARFILT(alpha,p,rvar,X,n,m, ier) 


Input: 
alpha - Array of length p containing coefficients a. 
Pp - Integer containing order p (>0). 
rvar - Real scalar containing error variance a? (>0). 
rs - Array containing the (nxm) matrix to be filtered. 
n,m - Integers containing the size of X. 
Output 
¥ - Array of length nm containing the filtered version of X. 
ier - Integer variable indicating whether or not the AR process is 


stationary (0 means yes, anything else means no). 
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Remarks: 
1. If ier# 0, then Y is not formed. 
2. Workspace=2nm+p?+2p. 


ARMACORR Command 


Purpose: 


Calculate the variance R(0) and autocorrelations p(1),...,o(M) of an 
ARMA(p, q,@, B, 0”) process given its parameters. 


Format: 


rho=ARMACORR (alpha, beta,p,q,rvar,M,RO,ier) 


Input: 
alpha - Array of length p containing coefficients a. 
beta - Array of length q containing coefficients f. 
P - Integer containing order p (>0). 
q - Integer containing order g (>0). 
rvar - Real scalar containing error variance a? (>0). 
M - Integer (>max(p,q)) containing the number of autocorrela- 
tions to calculate. 
Output 
RO - Real scalar containing variance R(0) of process. 
ier - Integer variable indicating whether or not the ARMA process 
is stationary (0 means yes, anything else means no). 
rho - Array of length M containing autocorrelations p(1),..., p(M). 
Remarks: 


1. If ier=1, then RO and rho are not calculated. 
2. If M=0, then RO is calculated but rho is not. 
3. Workspace=p+qtM+1+2 (max (p,q)+1). 
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a PEED 


ARMADT Command 


ee ee 


Purpose: 


Generate a realization of length n from a Gaussian ARMA(p, q, @, B, 0”) 


process X. 


Format: 


X=ARMADT (alpha, beta,p,q,rvar ,seed,n,M,rho,r0,ier) 


Input: 
alpha 


beta 


Remarks: 


Array of length p containing coefficients a. 
Array of length q containing coefficients #. 
Integer containing order p (>0). 

Integer containing order q (>0). 

Real scalar containing error variance a? (>0). 


Real scalar containing the seed for the random number gen- 
erator. 


Integer (>max (p,q) ) containing the length of the realization. 


Integer containing the number of autocorrelations to calculate 
(>max(p,q)). 


Array of length M containing autocorrelations p(1),...,p(M). 
Real scalar containing variance R(0) of process. 


Integer variable indicating whether or not the ARMA process 
is stationary (0 means yes, anything else means no). 


Array of length n containing realization. 


1. If ier=1, then rho,RO, and X are not calculated. 


2. Workspace=p+qtr?+5M+2n, where r=max(p,q+1). 
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Purpose: 
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ARMAPRED Command 


Find minimum mean square error linear predictors and (optionally) pre- 
diction standard errors for an ARMA(p, q, @, 8,07) process X. 


Format: 


ARMAPRED(X,n,alpha,beta,p,q,rvar,ti,t2,h1,h2,XP,ier[,SE]) 


Input: 


SE 


Array of length n containing the realization to be used in the 
prediction. 


Integer containing the length of the realization (>0). 
Array of length p containing coefficients a. 

Array of length q containing coefficients B. 

Integer containing order p (>0). 

Integer containing order q (>0). 

Real scalar containing error variance 0” (>0). 


Integers (1<ti<t2<n) specifying the range of memories to 
be used (see Remark 1 below). 


Integers (1<h1<h2) specifying the horizons to be used (see 
Remark 1 below). 


Integer variable indicating whether or not the ARMA process 
is stationary (0 means yes, anything else means no). 


Array of length (t2—t1+1)(h2—h1+1) containing predictors 
(see Remark 1 below). 


If this argument is present, it contains the prediction standard 
errors for the predictors in the array XP. 
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1. The memory-t, horizon-h predictor and prediction standard error of X (t+ 
h) given X(1),..., X(t) are found for t between t1 and t2, for h between 
hi and h2, and placed into the arrays XP and SE (SE is only formed if 
present in the argument list) . The values for t=t1 come first, then come 
the values for t=t1+1, and so on. 


2. Workspace=n+6r?+6r+j*npreds, where r=max(p,q+1) and j is 1 if the 
SE argument is not present, and j is 2 if it is present. 


_— SS 558 SE a 


ARMASEL Command 


SS 


Purpose: 


Do a stepwise ARMA analysis of data X(1),..., X(n). 


Format: 


ARMASEL(X,n,M,s,k1,k2,kopt,pval »P,q,alpha, beta,rvar,ier) 


Input: 
X 


n - 


pval = 


Array of length n containing the time series to be analyzed. 


Integer containing the length of the data to be analyzed. If 
n is negative, no output is displayed and the sample size is 
taken as the absolute value of n. 


Integers determining the maximum AR and MA lags that can 
be included in the model. An AR model of length s, s<M is 
fit to X and then the largest AR and MA lags that can be 
included are M—s. Thus 1 <M—s < 40. 


Integers giving the number of AR and MA lags that can be 
included in the model (0<k1,k2<M-s, see Remark 1 below). 


Integer determining whether ARMASEL should choose the num- 
ber of lags and what they are (kopt=0), whether the user will 
specify the number of lags and ARMASEL will choose what they 
are (kopt>0), or whether the user will specify both the num- 
ber of lags and what they are and ARMASEL will only estimate 
the coefficients (kopt<0). See Remark 1 below for details. 


Real scalar specifying the p-value for inclusion at each step of 
the stepwise procedure (0<pval<1). Note that a small value 
for pval makes it easier for a lag to be included. 
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alpha 


ier 


Remarks: 
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(If kopt<0 and k1>0) Array of length k1 specifying the AR 
lags to be forced into the model. 


(If kopt<0 and k2>0) Array of length k2 specifying the MA 
lags to be forced into the model. 


Integer variable containing the order of the AR part of the 
chosen ARMA model. Thus p is the maximum AR lag chosen. 


Integer variable containing the order of the MA part of the 
chosen ARMA model. Thus q is the maximum MA lag cho- 
sen. 


(If p>0) Real array of length p containing the coefficients for 
the AR lags that were chosen and zeros for those not chosen. 


(If q>0) Real array of length q containing the coefficients for 
the MA lags that were chosen and zeros for those not chosen. 


Real scalar containing the estimate of the error variance. 


Integer indicating if any numerical errors were encountered (0 
means no, 1 means the matrix being swept becomes singular). 


1. If kopt=0, then the ki AR lags and k2 MA lags out of the M-s possible 
lags having highest correlation in absolute value with X(t) are possible 
contenders for inclusion in the model and ARMASEL will continue adding 
and deleting lags until the stepwise procedure says to stop. 


2. If kopt=j>0, then the same lags as in Remark 1 are contenders for inclu- 
sion, but ARMASEL will force into the model the “best” j lags, that is, the 
lags chosen by the stepwise procedure. 


3. If kopt=j<0, then the —j = kl + k2 lags inputted in the arrays alpha 
and/or beta are forced into the model. 


4. Workspace=2M+4 (M-s)?+700. 


ARMASP Command 


Purpose: 


Calculate the spectral density of an ARMA(p, q, a, B, 0”) process. 
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Format: 


f=ARMASP (alpha, beta,p,q,rvar,Q) 


Input: 

alpha - Array of length p containing coefficients a. 

beta - Array of length q containing coefficients £. 

Pp - Integer containing order p (>0). 

Q - Integer (>max(p,q)) containing the number of frequencies 
between 0 and 1 at which to calculate the spectral density. 
The greatest prime factor of Q can be at most 23, and the 
product of its square-free prime factors can be at most 210. 

rvar - Real scalar containing error variance 0? (>0). 

Q - Number of frequencies between 0 and 1 at which to calculate 
spectra (the greatest prime factor of Q must be <23, and the 
product of its square-free prime factors must be <210). 

Output: 

f - Array of length m =[Q/2]+1 containing the values of the 

spectral density at the frequencies (j — 1)/Q, 7 =1,...,m. 
Remark: 


1. Workspace=2Q. 


ARPART Command 


Purpose: 


Calculate the partial autocorrelations of an AR(p,a@, a”) process given its 
coefficients. ARPART can also be used to check coefficients for having character- 
istic polynomial with zeros outside the unit circle. 


Format: 
part=ARPART (alpha,p, ier) 
Input: 
alpha - Array of length p containing coefficients a. 


p - Integer containing order p (>0). 
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Output: 


part - 


ier - 
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(If ier=0) Array of length p containing partial autocorrela- 
tions. 


Integer variable indicating whether the zeros of the character- 
istic polynomial corresponding to alpha are all outside the 
unit circle (0 means they are, anything else means they are 
not). 


ARSP Command 


ee 


Purpose: 


Calculate the spectral density of an AR(p,a,o”) process. 


Format: 


f=ARSP (alpha, p,rvar,Q) 


Input: 
alpha - 


rvar = 


Output: 


Remark: 


1. Workspace= 


Array of length p containing coefficients a. 
Integer containing order p (>0). 
Real scalar containing error variance o? (>0). 


Integer (>p) containing the number of frequencies between 0 
and 1 at which to calculate the spectral density. The greatest 
prime factor of Q can be at most 23, and the product of its 
square-free prime factors can be at most 210. 


Array of length m =[Q/2]+1 containing AR spectra at the 
frequencies (j — 1)/Q, j =1,...,m. 


2Q. 
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ee ee 


ARSP2 Command 


SS eee eee 


Purpose: 


Calculate auto- and cross-spectra for a bivariate autoregressive process of 


order p. 


Format: 


ARSP2(A,p,sigma,Q,fxx »fyy,fxyr,fxyi) 


Input: 


sigma 


Output: 
£xx 
fyy 
fxyr 
fxyi 


Remarks: 


Three-dimensional array containing AR coefficient matrices 
for order p. 


Integer containing order p (>0). 
Array of length 4 containing error covariance matrix. 


Integer containing the number of frequencies between 0 and 
1 at which to calculate spectra. ARSP2 does not use the fast 
Fourier transform so there are no restrictions on the factors 
of Q. 


Array containing autospectra of first series. 
Array containing autospectra of second series. 
Array containing the real part of the cross-spectra. 


Array containing the imaginary part of the cross-spectra. 


1. fxx,fyy,fxyr,fxyi are each real arrays of length m =[Q/2]+1 containing 
values for frequencies (j — 1)/Q, 7 =1,...,m. 


2. p, A, and sigma will be in the correct form if they are obtained from 


CORRAR2. 


3. Workspace=4 (p+1)+4[Q/2]+1. 
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eee 


ARSPCB Command 


Purpose: 
Find confidence bands for the spectral density of an AR(p, a, a7) process. 


Format: 


ARSPCB(alpha,p,rvar,n,Q,conf,f1,fu) 


Input: 

alpha - Array of length p containing coefficients a. 

Pp - Integer containing order p (>0). 

rvar - Real scalar containing error variance a? (>0). 

n - Integer containing the length of the realization that was used 
to estimate the coefficients (>0). 

Q - Number of frequencies between 0 and 1 at which to calculate 
the bands (the greatest prime factor of Q must be <23, and 
the product of its square-free prime factors must be <210). 

conf - Real scalar containing the desired confidence level for the 
bands (0<conf<1). 

Output: 

fl,fu  - Arrays of length m =[Q/2]+1 containing the lower and up- 
per confidence bands each at the frequencies (j — 1)/Q, 7 = 
lps cenit 

Remark: 


1. Workspace=5 (p+1)+3 (p+1)?+3Q. 


ARSPPEAK Command 


Purpose: 


To find the frequency corresponding to a peak in the spectral density of 
an AR(p,a@, a7) process. 
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ARSPPEAK (alpha, p,rvar,n,ier »wp,se[,start]). 


Input: 
alpha 


rvar 


start 


Output: 


ier 


Wp 


se 


Remarks: 


Array of length p containing coefficients a. 
Integer (1 < p < 55) containing the order p. 
Real scalar containing error variance a? (>0). 


Integer containing (if ARSPPEAK is being used for estimation 
purposes) the length of the realization that was used to es- 
timate the parameters of the process. If the parameters are 
the true values, let n=1. 

An optional argument that is a real scalar containing a start- 
ing value (0<start<.5) for the maximum finding procedure. 
If start is not present, then ARSPPEAK uses the largest rel- 
ative maximum of the spectral density evaluated at 256 fre- 
quencies between 0 and 1 as the starting value. 


An integer indicating whether or not ARSPPEAK was successful 
in finding a peak. The possible values of ier are: 


0 ARSPPEAK was successful in finding a peak. 


1 ARSPPEAK judged that the AR spectral density has 
no relative maxima. 


2 A zero second derivative was encountered. 
3 The maximum finder converged to frequency 0 or .5. 
4 The maximum finder didn’t converge. 

(If ier=0) The peak frequency. 


(If ier=0) The standard error of the peak frequency estima- 
tor. 


1. Using different starting values will allow more than one peak to be found 
if there are more than one. 


2. Workspace=3(ptp?). 
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BARTTEST Command 


Purpose: 


Perform Bartlett’s test for the significance of the maximum deviation of a 
cumulative spectra from the spectral distribution of white noise. 


Format: 
1. pval=BARTTEST(F,Q,n,b) 
2. pval=BARTTEST (b) 


Input: 
F - Array of length [Q/2]+1 containing the values of a cumulative 
spectra (usually produced by CUMSP). 
Q - Integer containing the number of frequencies at which original 
spectra was calculated. 
n - Integer containing the sample size to be used in the test. 
Output: 
b - Real scalar variable containing the value of the test statistic. 
pval - Real scalar variable containing the p-value corresponding to 
the b; that is, pval is the probability (under the hypothesis 
of white noise) of observing a deviation of size b or more. 
Remarks: 


1. The value of [Q/2]+1 must equal the length of F. 


2. In the second form, b is input. 


BATCHOFF Command 


Purpose: 


Cancel the effect of BATCHON. 


Format: 


BATCHOFF 
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Remark: 


1. While BATCHON is in effect, all variables are automatically overwritten if 
required. When BATCHOFF is issued, the OVERON/OVEROFF status takes over 
again. 


An cincwcmeameeE ee ee ee ee 
BATCHON Command 


ee eee 


Purpose: 

Put TIMESLAB into “batch mode,” that is, TIMESLAB does not pause 
during lists or plots. 
Format: 


BATCHON 


Remark: 


1. BATCHON has the effect of OVERON until BATCHOFF is issued. 


Se ee eee 
BINOM Command 


SS eee 


Purpose: 


Find binomial coefficients and probabilities. 


Format: 
1. x=BINOM(n,k) 
2. x=BINOM(n,p) 
3. x=BINOM(n,k,p) 


Remarks: 


1. In the first form, the output real scalar x contains the binomial coefficient 
(n choose k) for the input integers n and k. 


2. In the second form, the output array x of length n+1 contains the binomial 
probabilities for 0,1,...,n successes for n trials and success probability p 
where n and p are input and are an integer and a real scalar respectively. 
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3. In the third form, the output real scalar contains the probability that a 
binomial random variable having n trials and success probability p is equal 
to k. n and k are input integers and p is an input real scalar. 


4. Workspace=n+1 in the second form. 


CLEAN Command 


Purpose: 


Delete variables. 


Format: 

CLEAN - TIMESLAB asks users about each of the arrays that 
have been defined, giving them the option of either 
deleting it, not deleting it, or quitting the prompting. 

CLEAN(list) - Each of the variables in the list will be deleted. 

CLEAN (all) - All of the variables that have been defined will be 
deleted. 

CLEAN(arrays) - Allof the arrays that have been defined will be deleted. 

CLEAN(reals) - All of the real scalar variables that have been defined 
will be deleted. 

CLEAN(ints) - All of the integer variables that have been defined will 
be deleted. 

CLEAN(chars) - All of the character variables that have been defined 
will be deleted. 

CLS Command 
Purpose: 


Clear the screen in either the text or graphics mode. 


Format: 


CLS 
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COEFFCSD Command 


— eee ee eee 


Purpose: 


Find the asymptotic standard deviation of estimates of AR, MA, or ARMA 


processes. 
Format: 

aie COEFFCSD (coeff ,p,n,SD) 

2. COEFFCSD (alpha, beta, P,4,n,S5DALPH , SDBETA, ier) 
Remarks: 

1. The first form is used for AR or MA processes. coeff,p, and n are the 
coefficients, order, and sample size and are input, while SD is an output 
array of length p containing the asymptotic standard deviations of the 
coefficients. 

2. The second form is for ARMA processes. alpha,beta,p,q, and n are 
the coefficients, orders, and sample size. The output arrays SDALPH and 
SDBETA of lengths p and q are the asymptotic standard errors for the AR 
and MA coefficients respectively, while the output integer variable ier 
is an error indicator (0 means no error, 1 means a singular matrix was 
encountered). 

3. Workspace=p*+p for the first form, and 2r?+r+p+q in the second form 


where r=2max (p,q) +1. 


SS ee ee ee ee 


COLOR Command 


a oe ee 


Purpose: 


Set the foreground color to be used henceforth in the graphics mode. 


Format: 
COLOR (kolor) 
Input: 
kolor - An integer determining the color to be used according to the 


rules given below. 
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Remarks: 


1. On a PC with a color graphics adapter (CGA), the background color is 
always black, while the foreground color can be any one of the 15 colors in 
the table given below (plus kolor=0 which gives a black foreground and 
thus should not be used). To set the foreground color, enter the command 
COLOR(kolor) where kolor is one of the 15 values in the table. 


2. On a PC with an enhanced graphics adapter (EGA), the foreground color 
can be either green or blue-green, and the background color can be any 
one of the 15 colors in the table given below (plus kolor=0 which gives 
a black background). To set the foreground color to green or blue-green, 
enter the command COLOR(256) or COLOR(257) respectively. Then to set 
the background color, enter the command COLOR(kolor), where kolor is 
0 for black or any one of the 15 colors in the table. 


3. TIMESLAB starts with yellow on black for a CGA and green on brown 
for an EGA. 


Foreground Colors in the Graphics Mode 


kolor Color kolor Color kolor Color 
1 Blue 6 Yellow-brown ta Light blue-green 
u Green 7 White 12 Light red 
3 Blue-green 8 Gray 13 Light purple 
4 Red 9 Light blue 14 Yellow 
5 Purple 10 Light green 15 Bright white 


CORR Command 


Purpose: 

Calculate the variance and/or correlogram and/or sample spectral density 
of a univariate time series data set. 
Format: 


rho=CORR(X,n,M,Q,iopt,RO,f) 


Input: 
x - Array of length n containing data. 
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iopt 


rho 


Remarks: 
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Integer containing the length of the data (>0). 


Integer containing the number of correlations to calculate 
(O<M<n). 


Integer containing the number of frequencies in [0,1] at which 
to calculate the sample spectral density. The largest prime 
factor can be at most 23, while the product of the square-free 
prime factors can be at most 210. 


Integer indicating whether the sample mean should be sub- 
tracted before doing the calculations (1 means yes, 2 means 
no, the array X is not affected either way). 


Real scalar variable containing the sample variance (always 
calculated). 


Array of length m =[Q/2]+1 containing the sample spectral 
density at the frequencies (j — 1)/Q, j =1,...,m (only cal- 
culated if Q>n+M). 


Array of length M containing correlations (only calculated if 
M>0). 


1. If M=0, then RO is calculated but rho is not. 
. If Q=0, then f is not calculated. 


2 
3. If both n and M are positive, then Q must be at least n+M. 
4 


. Workspace=n+2Q+M. 


SS 


CORR2 Command 


ee 


Purpose: 


Calculate sample autocorrelations and cross-correlations for two univariate 


time series. 


Format: 


CORR2(X,Y,n,M, iopt ,RXO,RYO, rhoxy0,rhox,rhoy, rhoxy , rhoyx) 


rhoxy0 


rhox 


rhoy 


rhoxy 


rhoyx 


Remark: 
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Array of length n containing the first series. 

Array of length n containing the second series. 
Integer containing the length of the data sets. 
Integer (<n) containing the number of lags desired. 


Integer indicating whether or not to subtract the sample 
means before doing the calculations (1 means yes, 2 means 
no, the arrays X and Y are not effected in either case). 


Real scalar containing sample variance of the first series (>0). 


Real scalar containing sample variance of the second series 
(>0). 


Real scalar containing sample cross-correlation of lag 0. 


Array of length M containing autocorrelations of lags 1,...,M 
for first series. 


Array of length M containing autocorrelations of lags 1,...,M 
for second series. 


Array of length M containing cross-correlations of lags 1,..., M. 


Array of length M containing cross-correlations of lags —1 
Rhrough —M. 


1. Workspace=2n+4M. 


CORRAR Command 


Purpose: 


Calculate the coefficients and error variance of an AR(p,a@,o”) process 
given its variance and correlations. In the first form the order is specified, 
while in the second form the order is chosen by the CAT criterion. 


Format: 


1. alpha=CORRAR(rho,RO,p,rvar) 


2. alpha=CORRAR(rho,RO,M,n,p,rvar,CAT) 
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Input: 
rho - Array of length p (first form) or M (second form) containing 
autocorrelations. 
RO - Real scalar containing sample variance (>0.). 
- (First form) Integer containing the AR order (>0). 
M - (Second form) Integer containing the largest order to fit (>0). 
n - (Second form) Integer containing the sample size. 
Output 
p - (Second form) Integer variable containing the order chosen. 
rvar - Real scalar variable containing error variance. 
alpha - Array of length p containing AR coefficients. 
CAT - (Second form) Array of length M containing values of CAT for 
orders 1,...,M. 
Remarks: 


1. If p=0 in the second form, then rvar and alpha are not formed. 


2. Workspace=3p for the first form, and 3M for the second form. 


a  * 
CORRAR2 Command 


SSS ee ee eee 


Purpose: 


Calculate bivariate AR parameters given variances and correlations. 


Format: 
A=CORRAR2(RXO,RYO, rhoxy0,rhox,rhoy, rhoxy ,rhoyx,M,iopt, 
n,p,SIGMA,CAT, ier) 


A=CORRAR2(RXO,RYO, rhoxy0,rhox,rhoy, rhoxy ,rhoyx,p, Sigma, ier) 


Input: 
RXO - Real scalar containing sample variance of the first series (>0). 
RYO - Real scalar containing sample variance of the second series 


(>0). 
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rhoxyO - Real scalar containing sample cross-correlation of lag 0. 

rhox - Array of length M containing autocorrelations of lags 1,...,M 
for first series. 

rhoy - Array of length M containing autocorrelations of lags 1,...,M 
for second series. 

rhoxy - Array of length M containing cross-correlations of lags 1,...,M. 

rhoyx - Array of length M containing cross-correlations of lags —1 
through —M. 

M - Integer (< 200) containing the order to fit (if iopto is 1) or 


the maximum order to fit (if ioto is 2). 


iopto - Integer indicating what AR order(s) to fit. The following 
values are available: 


1 Fit order M. 
2 Fit CAT determined order <M. 


n - Integer containing sample size for use in CAT (>0). 
Output: 

p - Integer variable containing the AR order that was fit. 

SIGMA - Array of length 4 containing error covariance matrix. 

CAT - Array of length M containing the CAT criterion for orders 
dee aes Me 

ier - Integer variable indicating possible errors (0 means no, 1 
means yes). 

A - Three-dimensional array containing AR coefficient matrices 


for order p (not formed if p=0). 


Remarks: 
1. The second form can be used when the user knows what order to use. 
2. The bivariate Levinson’s algorithm is used. 


3. ier=1 indicates either that the data are too ill-conditioned or that the 
user is trying to fit more parameters than there are observations (2n must 
be > 4M+3). 


4. Workspace=42M+4. 
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ae es ee lee een le ee eee 
CORRARMA Command 
ee CE 


Purpose: 

Calculate the coefficients and error variance of an ARMA(p, q, @, B, o”) 
process given its variance and autocorrelations. 
Format: 


rvar=CORRARMA (rho, RO »P,q,maxit,del,ier,alpha, beta) 


Input: 
rho - Array of length p+q containing the autocorrelations of the 
process. 
RO - Real scalar containing the variance of the process (>0.). 
Pp - Integer containing order p (>0). 
q - Integer containing order q (>0). 
maxit - Integer containing the maximum number of iterations to allow 
in Wilson’s algorithm (see Remark 1 below). 
del - Real scalar containing convergence criterion (>0). 
Output: 
ier - Integer variable containing an error/convergence indicator. 
The following values are possible: 
Q CORRARMA successfully found the ARMA parameters. 
1_ A singular matrix was encountered trying to find AR 
parameters. 
2  Wilson’s algorithm for finding the MA parameters 
didn’t converge. 
alpha - Array of length p containing AR coefficients. 
beta - Array of length q containing MA coefficients. 
rvar - Real scalar variable containing the error variance o?. 
Remarks: 


1. CORRARMA finds alpha by solving the high-order Yule-Walker equations. 
This system of equations can be singular in which case ier is set equal to 
1 and the output quantities are not formed. 
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2. If finding alpha causes no error, then the correlations of the MA part of 
the model are calculated and Wilson’s algorithm is used to convert them 
to beta and rvar. This iterative algorithm is not guaranteed to converge 
in the maxit iterations allowed by the user. If it doesn’t converge, ier is 
set equal to 2, and beta and rvar are not formed. A message is displayed 
upon nonconvergence. 


3. Workspace=14(p+q)+(p+q)?. 


CORRMA Command 


Purpose: 


Calculate the coefficients and error variance of an MA(q,B,o7) process 
given its autocorrelations. 


Format: 


beta=CORRMA (rho,q,RO,maxit,del,ier,rvar) 


Input: 


rho 


rvar 


beta 


Remarks: 


Array of length q containing autocorrelations of lags 1,... , q. 
Integer containing order g (>0). 
Real scalar containing the variance of the MA process. 


Integer containing the maximum number of iterations to use 
in Wilson’s algorithm (>0). 


Real scalar containing the convergence criterion to use in 
Wilson’s algorithm (>0). 


Integer variable indicating whether or not Wilson’s algorithm 
converged (0 means yes, 1 means no). 


Real scalar variable containing the error variance o” of the 
MA process. 


Array of length q containing MA coefficients. 


1. If the algorithm doesn’t converge in maxit iterations, TIMESLAB displays 
a message giving the difference in consecutive iterations. 
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2. If ier is 1, then rvar and beta are not formed. 


3. Workspace=10q. 


i 
COS Command 


a eee 


Purpose: 


Calculate cosines of various kinds. 


Format: 
1. y=COS (x) 
2. Y=COS(X,n) 
3. Y=COS(n,a,p) 


Remarks: 


1. In the first form, the output real scalar variable y is the cosine of the real 
scalar input (expressed in radians) x. 


2. In the second form, the output array Y of length n contains the cosines of 
the corresponding values in the input array X, which must be expressed in 
radians. 


3. In the third form, the elements of the output array Y of length n (n is 
an input integer) are given by Y(i)= acos(2z(i — 1)/p), where the real 
scalars a and p are input and p¥0. 


Ee eee ee 
CROSSP Command 


SS ee eee 


Purpose: 


Calculate windowed cross-spectra for two time series given their cross- 
correlations. 


Format: 
CROSSP (rhoxy , rhoyx,RX0,RYO, rhoxy0,M,Q,iopt,fxyr,fxyi) 
Input: 


rhoxy - Array of length M containing cross-correlations of lags 1,...,M. 


iopt 


Output: 
fxyr 


fxyi 


Remark: 
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Array of length M containing cross-correlations of lags —1 
through —M. 

Real scalar containing sample variance of the first series (>0). 


Real scalar containing sample variance of the second series 
(>0). 


Real scalar containing sample cross-correlation of lag 0. 


Integer containing the truncation point to be used (0 < M <Q, 
must be even for iopt=6). 


Number of frequencies between 0 and 1 at which to calculate 
spectra (the greatest prime factor of Q must be <23, and the 
product of its square-free prime factors must be <210). 


Integer indicating the window to use: 
1 ‘Truncated periodogram 

2 Bartlett 

3 Tukey 

4 Parzen 

5 


Bohman 


Array of length m =[Q/2]+1 containing the real part of the 
cross-spectra at frequencies (j — 1)/Q, 7 =1,...,m. 


Array of length m =[Q/2]+1 containing the imaginary part 
of the cross-spectra at frequencies (j — 1)/Q, 7 =1,...,m. 


1. Workspace=m+3Q. 


Purpose: 


CUM Command 


Calculate cumulative sums or averages of the first n elements of an array. 


Format: 


Y=CUM(X,n,iopt) 
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Input: 
X - Array of length at least n containing the input data. 
n - Integer containing the number of sums or averages to find. 
iopt - Integer indicating whether cumulative sums (iopt=1) or cu- 
mulative averages (iopt=2) are desired. 
Output: 
Y - Array of length n containing cumulative sums or averages. 
Remark: 


1. Workspace=n. 


ee eS ee ee 
CUMSP Command 


eee eee eee 


Purpose: 


Calculate cumulative spectra given a spectral array. 


Format: 


F=CUMSP (£ ,Q) 


Input: 
£ - Array of length m =[Q/2]+1 containing the values of a spec- 
tral density at the frequencies (j — Ly) Oe ee oye. 
Q - Integer containing the number of frequencies at which £Y was 
calculated. 
Output: 
F - Array of length m =[Q/2]+1 containing values of the cumu- 
lative spectra at the frequencies (j — 10,9 = lem, 
Remark: 


1. Workspace=[Q/2]+1. 
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DELAY Command 


Purpose: 


Cause TIMESLAB to delay execution for a specified amount of time, usu- 
ally in a macro. 


Format: 
DELAY (t) 
Input: 
is - Real scalar containing the amount of time (in seconds) to 
delay (>0.). 
DENSITY Command 
Purpose: 


Find a kernel probability density estimator for a data set. 


Format: 


DENSITY(X,n,rbins,M,kernel,Y,FY) 


Input: 

X - Array of length n containing data. 

n - Integer containing the length of X. 

rbins - Real scalar determining the bandwidth to be used (see Re- 
mark 1). 

M - Integer containing the number of points at which to estimate 
the density. If M is negative, then the values at which the 
density is to be estimated are entered in the array Y, and the 
number of such values is taken to be the absolute value of M. 

¥ - (If M is negative) An array containing the values at which 
the density is to be estimated. The elements of Y must be in 
increasing order. 

kernel - Integer containing which kernel to use. The following choices 


are available: 
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1 Rectangular 
2 Cosine 
3 Epanechnikov 
4  Biweight 
5 ‘Triangular 
6 Gaussian 
7 Parzen 
Output: 
ug - Array of length M containing values where density is esti- 
mated. 
FY - Array of length M containing values of the density estimator 
at the points in Y. 
Remarks: 


1. The bandwidth that is used by the DENSITY command is given by 


rT. 
nice = : rbins > 0 
ae rbins 


1.758szn-", rbins = 0) 


where r, and sz are the range and standard deviation of the data respec- 
tively, and q is 1/3 for the rectangular window and 1 /5 otherwise. 


2. Workspace=n+2M. 


a ee ee 
DIFF Command 


Se ee eee 


Purpose: 


Form an array that contains the differences of another array for a specified 
lag. 


Format: 
Y=DIFF (n,d,X) 
Input: 


n - Integer containing the length of the data. 
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d - Integer containing the lag to use (0<d<n). 
X - Array of length n containing the data to be differenced. 
Output: 
Y - Array of length n-d containing the result of the differencing X, 
that is, the ith element of Y is Y(i)=X (itd) -X(i), i=1,....n- 
d. 


DIST Command 


Purpose: 


Calculate gamma function (form 1) or pdf, cdf, or quantiles of Z, t, x”, or 
F distribution (form 2). 


Format: 
1. y=DIST(n,a,b) 
2. y=DIST(name, iopt ,n,x[,df1,df2]) 


Input: 
name - A single character specifying which distribution to use. The 
following choices are available: 
Zorz Use the standard normal distribution. 
Tort Use the Student-t distribution with df1 degrees of 
freedom. 
Corc Use the x? distribution with df1 degrees of freedom. 
Forf Use the F distribution with df1 and df2 degrees of 
freedom. 
iopt - An integer specifying which function is to be evaluated. The 
following choices are available: 
1 Evaluate probability density function. 
2 Evaluate cumulative distribution function. 
3 Evaluate quantile function. 
n - Integer containing the number of points at which to evaluate 


the function (n>0). 
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df1 


df2 


Remark: 
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An array (if n> 1) of length n containing the values at which 
to evaluate the specified function, or a real scalar (if n=1) 
containing the single value at which to evaluate the function. 


(Only in first form) Real scalars specifying at what points 
the function is to be evaluated. The n equally spaced points 
between a and b inclusive are the points to be evaluated. 


An integer containing the degrees of freedom for the t or x 
or the numerator degrees of freedom for the F (not needed if 
doing the standard normal distribution). 


An integer containing the denominator degrees of freedom for 
the F' distribution (not needed if not doing the F distribu- 
tion). 


If n=1, a real scalar variable containing the desired value. If 
n>1, an array of length n containing the desired values at the 
specified points. 


1. If the quantile function is desired, the values at which the function is to 
be evaluated must be in the open interval (0,1). 


eee 


DIVSDS Command 


Se a i 


Purpose: 


Calculate and/or divide and/or multiply seasonal standard deviations for 


a data set. 


Format: 


Y=DIVSDS(X,n,d,SDS[, iopt]) 


Input: 
x 
n 


d 


- Array of length n containing data to be operated upon. 
- Integer containing the length of X (>0). 


- Integer containing the period of the seasons (>0). 


SEC C.2 DETAILED DESCRIPTION OF COMMANDS 511 


iopt - Optional argument which if present is an integer indicating 
what operation is to be performed. The following options are 
available: 


0 Calculate seasonal standard deviations but don’t di- 
vide the data by them. 


1 Calculate and divide seasonal standard deviations. 


2 Multiply data by inputted SDS (this allows the in- 
verse operation of iopt=1). 


SDS - (Only if iopt=2) Array of length d containing values to mul- 
tiply data by. 
Output: 
Y - (Unless iopt=0) Array of length n containing the result of 
dividing the input data by the seasonal standard deviations. 
SDS - (Unless iopt=2) Array of length d (if d=1) or a real scalar 
variable (if d=1) containing seasonal standard deviations. 
Remarks: 


1. The argument iopt not being present is equivalent to iopt=1. 


2. Workspace=n+d. 


DOS Command 


Purpose: 
Allow the user to issue any command that can be issued at the DOS 
prompt. 
Format: 
DOS 
Remarks: 


1. The user’s machine must have enough RAM to hold both TIMESLAB and 
COMMAND.COM and whatever command is issued. Further, a copy of 
COMMAND.COM must be on whatever device the user boots from. If 
there is not enough RAM or TIMESLAB can’t find COMMAND.COM, 
TIMESLAB issues an appropriate message and reissues the prompt. 
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2. The DOS command and then DOS commands can be issued from within 


macros. Make sure to issue the EXIT command in the macro to return to 
TIMESLAB. 


ee eee ee 
DOT Command 


Se 


Purpose: 


Find the inner product of the first n elements of two arrays. 


Format: 


c=DOT(X,Y,n) 


Input: 
X - Array of length at least n. 
Ny - Array of length at least n. 
n - Integer containing the number of elements of X and Y to use 
in the inner product. 
Output: 
Cc - A real scalar variable containing the inner product of the first 
n elements of X and Y. 
Remark: 


1. X and Y can be the same array. 


ae ee a gS 


DOUBLE Command 


Se eee 


Purpose: 
Extend spectral arrays by symmetry about frequency .5. 
Format: 
DOUBLE (fr,fi,Q) 
Input: 
fr,fi  - Arrays of length m =[Q/2]+1 to be extended to length Q. 
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Q - Integer containing the desired length of the extended arrays. 
Output: 

fr,fi  - Arrays of length Q containing the extended arrays. 
Remarks: 


1. The extended values are given by 
fr(Q—i)=fr(i+2) and f1(Q-—i) = —fi(i+2) 


for i=0,...,[Q/2]—1. 
2. The value of [Q/2]+1 must equal the length of fr and fi. 
3. Workspace=2Q. 


DTAR Command 


Purpose: 


Fit autoregressive models to univariate data. 


Format: 


alpha=DTAR(X,n,M,iopto,ioptl,ioptm,p,RO,rvar[,CAT]) 


Input: 
X - Array of length n containing data set. 
n - Integer containing the length of the data set. 
- Integer containing either the AR order to use (if iopto=1) 
or the maximum AR order to use (if iopto=2). 
iopto - Integer specifying whether to use order M (iopto=1) or to use 
the Parzen CAT criterion to determine an order less than or 
equal to M (iopto=2). 
ioptl - Integer specifying whether or not to display on the screen 
a table of partial autocorrelations, “biased” and “unbiased” 
standardized residual variances, and the CAT criterion for 
orders 1 through M (1 means yes, 2 means no). 
ioptm ~- Integer specifying which method is to be used in doing the 


estimation. The following choices are available: 
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1 Use the algorithm in Theorem 3.4.5 to find Yule- 
Walker estimates (YWE). 


2 Use the Burg algorithm. 


3 Each of the M partial autocorrelations from the algo- 
rithm in ioptm=1 will be tested for significance (ver- 
sus +2/,/n) and the parameters of the fitted model 
are found from the partials judged to be significantly 
different from zero. 


4 This option is the same as 3 except the Burg algo- 
rithm is used. 


5 This option uses an AIC-based method for testing 
for significance the partial autocorrelations from the 
YWE algorithm. The method starts with no par- 
tials in the subset. At each successive step, the par- 
tial largest in magnitude of those not in the sub- 
set is added and AIC is calculated for the new sub- 
set. If this AIC is smaller than the previous one the 
method continues. Otherwise the last added partial 
is dropped and the method stops. From the deter- 
mined subset of partials, an AR order, coefficients, 
and error variance are determined. 


6 This option is the same as 5 except that the Burg 
algorithm is used to get the partials. 


Output: 
Pp - Integer variable containing the order of the chosen model. 
RO - Real scalar variable containing the sample variance of the 
data. 
rvar - Real scalar variable containing the error variance of the cho- 
sen model. 
CAT - If this argument is included, it is an array of length M con- 
taining the values of the CAT criterion for orders 1 through 
M. 
alpha - Array of length p containing the coefficients of the chosen 
model. 
Remarks: 


1. If p=0, then rvar and alpha are not formed. 


2. Workspace=2 (n+M) +3M. 
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DTARMA Command 


Purpose: 


Find exact maximum likelihood estimators of the parameters of an ARMA 
(or AR or MA) process. 


Format: 


DTARMA(X,n,p,q,maxit,eps,alpha,beta,rvar,m211,ier,del) 


Input: 

X - Array of length n containing data to be used in the estimation 
procedure. 

n - Integer containing the length of the data. 

Pp - Integer containing the AR order to use. 

q - Integer containing the MA order to use. 

maxit - Integer (1 < maxit < 500) containing the maximum number 
of iterations in the optimization procedure (see Remark 1). 

eps - Real scalar containing the convergence criterion (see Remark 
1)3 

alpha - Array of length p containing starting values for the AR coef- 
ficients (see Remark 2). 

beta - Array of length q containing starting values for the MA co- 
efficients (see Remark 2). 

Output: 

alpha - Array of length p containing the values of the AR coefficients 
when DTARMA finishes (see Remark 3). 

beta - Array of length q containing the values of the MA coefficients 
when DTARMA finishes (see Remark 3). 

rvar - Real scalar variable containing the value of the error variance 
when DTARMA finishes (see Remark 3). 

m211 - Real scalar variable containing the value of -2 times the log 


likelihood evaluated at the output values of the parameters. 
This is useful in calculating the AIC. 
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ier - Integer variable indicating the presence of errors and /or con- 
vergence. If ier is 0, then there were no errors encountered 
and convergence was reached. If ier is 1, no errors were en- 
countered but convergence was not reached. If ier is greater 
than 1, there was an error. This is usually because the model 
being estimated is very ill-conditioned (zeros of the AR or 
MA polynomials near the unit circle). Sometimes using dif- 
ferent starting values will fix the problem. If nothing seems to 
help, consistent estimates can be obtained using the ARMASEL 
command. 


del - A real scalar variable containing the sample variance of the 
values of the likelihood function at the vertices of the final 
simplex. This gives a measure of how close to convergence 
the algorithm has come. If this is less than eps, then DTARMA 
has come close to converging but not to a relative maxima. 


Remarks: 


i 
2. 


The orders p and q can’t both be zero. 


If p or q is zero, then the corresponding coefficient arrays are ignored 
although an argument must be included in the argument list. 


. If convergence has been reached, it is to relative maximum of the likeli- 


hood. To be safe, it is good to try various starting values to see if they all 
converge to the same place. If convergence has not been reached, DTARMA 
can be called again with the output coefficients as new starting values. 


. The Nelder-Mead simplex derivative-free optimization method is used with 


the Kalman Filter Algorithm to evaluate the likelihood. 


. Starting values can be obtained by CORRAR, CORRMA, or CORRARMA for AR, 


MA, and ARMA processes respectively. DTARMA will check the input coeffi- 
cients for stability. Both alpha and beta must give characteristic polyno- 
mials having all zeros outside the unit circle. DTARMA checks for this. The 
output alpha and beta are guaranteed to be stable. Sometimes just using 
zeros for starting values will work. Starting values can also be obtained 
from ARMASEL or SEASEST. 


- Workspace=3 (p+q) +2n+5 (max (p,q+1)+1)+(p+q) (p+q+1). 


SECIG2 


Purpose: 
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DTFORE Command 


Perform iterated AR forecasting of data (i.e., use a one lag, possibly non- 
stationary subset AR followed by a stationary full AR). 


Format: 


DTFORE(X,n,M1,M2,npr,lag,p,alphai, alpha2, XP) 


Input: 
X 
n 


Mi 


M2 


Array of length n containing data. 
Integer containing the length of the data (>0). 


Integer indicating what the first AR model should be. Values 
of M1 have the following meanings: 


j>0 The lag to use is the one less than or equal to M1 
having largest least squares regression coefficient in 
absolute value. 


j<O The lag to use is —j. 
0 The first AR is not done. 


Integer indicating what the second AR should be. The values 
of M2 have the following meanings: 


j>0 ‘The order used is the one less than or equal to j as 
chosen by the CAT criterion. 


j<O The order used is —}. 
0 The second AR is not done. 
Integer indicating how far into the future to predict (>0). 


Integer variable containing lag used for first AR. 
Integer variable containing order used for second AR. 
Real scalar variable containing the coefficient for the first AR. 


Array of length p containing coefficients for second AR. 
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XP - Array of length n+npr. The first lag+p elements are the first 
lagptp elements of X, the next n-lag-p elements are the one 
step ahead predictors of X(lag+p+1),...,X(n). The last npr 
elements of XP (if npr>0) are the 1 through npr step ahead 
predictors of the next npr values of the process. 

Remark: 


1. Workspace=4n+2|M1|+5|M2|+2npr. 


Sl 


ECHO Command 


— SS SS ee 


Purpose: 

To provide a command that does the same thing as striking the Ctrl-PrtSc 
keystroke, that is, to toggle the printer-echo feature of a RC: 
Format: 


ECHO 


Remarks: 


1. After ECHO is issued the first time, everything that appears on the screen 
in the text mode (except the list of commands given in response to the 
HELP command) also appears on the printer. This is called entering the 
printer-echo mode. Issuing ECHO again turns off the printer-echo. 


2. The PRINTSEL command must be issued first to tell TIMESLAB what 
printer is being used. 


ee ee ee 8 


EDIT Command 


ee ee ed ee eet 


Purpose: 


Invoke the TIMESLAB full screen text editor. 


Format: 
1. EDIT(oldfn,newfn) 
2. EDIT(fname) 
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Input: 
oldfn 


newfn 


fname 


Remarks: 
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The name of the file to be edited. If there is no file of this 
name, EDIT assumes the user wants to create a new file having 
name newfn. 


The name of the file to receive either the modified file or the 
newly created file. 


If the user wants the old and new files to be the same, then 
the second form of the command can be used. 


1. The file names can have at most 15 characters. 


2. See Section B.6 for more information about the EDIT command. 


Purpose: 


EIG Command 


Calculate the eigenvalues and (optionally) eigenvectors of a real symmetric 


matrix. 


Format: 


evals=EIG(A,n,maxit,ier[,evecs]) 


Input: 


An array of length n*n containing the matrix for which to 
find eigenvalues. 


Integer (<56) containing the size of the matrix. 


Integer (< 500) containing the maximum number of iterations 
to allow in the usual tridiagonalization and Q-R eigenvalue 
algorithm (usually 100 iterations is more than enough). 


Integer indicating whether convergence has been reached (0 
means it has, 1 means it has not). 


Array of length n containing the eigenvalues in decreasing 
order. 
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evecs - If this argument is present, it is an array of length n*n con- 
taining the eigenvectors of the matrix. The first n elements 
consist of the eigenvector corresponding to the first element of 
evals, the next n are the eigenvector for the second element 
of evals, and so on. 
Remark: 


1. Workspace=3n?+5n. 


Se ee ee 
END Command 


SSS eee eee 


Purpose: 


Provide a terminator for a WHILE...END construct in a macro. 


Format: 


END 


Remark: 


1. See Section B.6 for more information on the looping constructs available 
in TIMESLAB macros. 


i a. 
ENDIF Command 


a ee 


Purpose: 


Provide a terminator for an IF...ENDIF construct in a macro. 


Format: 


ENDIF 


Remark: 


1. See Section B.6 for more information on the looping constructs available 
in TIMESLAB macros. 
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ERASE Command 


Purpose: 


Erase a part of the screen in the graphics mode. 


Format: 
1. ERASE(cloc,rloc,ncs,nrs) 
2. ERASE(kode) 


Remarks: 


1. In the first form, a rectangle is erased that is ncs columns by nrs rows 
in size and whose lower left-hand corner is located at the pixel in column 
cloc and row rloc. 


2. In the second form, certain standard parts of the screen can be erased as 
specified by the following values of kode: 


0 The whole screen. 

The upper left quarter of the screen. 
The lower left quarter of the screen. 
The upper right quarter of the screen. 
The lower right quarter of the screen. 
The left half of the screen. 

The right half of the screen. 


SN DO oO Ff WO DH 


The top half of the screen. 
8 The bottom half of the screen. 


3. The screen has 640 columns and 200 rows of pixels. The columns are 
numbered from 0 to 639 from left to right, while the rows are numbered 
from 0 to 199 from bottom to top. 


EXP Command 


Purpose: 


Calculate the exponential function. 
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Format: 
1. y=EXP (x) 
2. Y=EXP(X,n) 


Remarks: 
1. In the first form, the input x and the output y are real scalars. 


2. In the second form, X is an input array of length at least n, and Y is an 
output array of length n. 


3. The largest value that can be exponentiated is 87.5. 


es ee 
EXTEND Command 


SS SS ee ee eee 


Purpose: 


Extend a data set by the inverse operation of differencing. 


Format: 


Y=EXTEND (X ,n,next,di[,d2]) 


Input: 
X - Array of length n containing data. 
n - Integer containing the length of X (>di+d2). 
next - Integer containing the number of points to extend X (>0). 
di,d2  - Integers containing differences. 
Output: 
‘Yi - Array of length ntnext containing X and the extended values. 
Remarks: 


1. The first n values of Y are the same as the first n values of X, while the 
next next values of Y are then found as described in Remarks 2 and 3. 


2. If d2 is zero, then Y(n+i)=Y (n+i-d1)+ybar, where ybar is the average of 
the di differences of the first n X’s. 


3. If d240, then Y(n+i)=Y (n+i-d1)+¥ (n+i-d2)-Y (n+i-d1-d2)+zbar, where 
zbar is the sample mean of the data resulting from taking the d1 and d2 
differences of the array X. 
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4. Workspace=ntnext. 


EXTRACT Command 


Purpose: 


Form an array by extracting elements of another array. 


Format: 
1. Y=EXTRACT(X,n1,n2) Y consists of X(n1) through X(n2). 


2. Y=EXTRACT(X,n1,n2,inc,ny) The output array Y consists of every 
incth element of X(m1) through X(n2). ny is the resulting number of 
elements in Y. 


3. Y=EXTRACT(X,XI,ni) The output array Y consists of the elements of 
X having indices contained in the array XI of length ni. 


4. Y=EXTRACT(X,n,r,relation,ny) The output array Y consists of the 
indices of the elements in the array X that satisfy a certain relation with 
the real scalar r. The relation argument can be one of 1t,le,eq,ge, or 
gt. ny is an integer variable containing the resulting number of elements 
in. Y/. 


5. B=EXTRACT(A,ra,ri,ci,nr,nc) The output array B is the nr by nc 
matrix consisting of the block of the input matrix A (which has ra rows) 
starting with its element in the r1 row and ci column. 


6. B=EXTRACT(A,ra,ri,r2,ic,nc) This form is the same as form 5 ex- 
cept that the nc columns of A having indices in the input array ic are 
extracted starting at row r1. 


Remark: 


1. For the first form, if only one point is extracted, then Y is a real scalar, 
otherwise it is an array. For the other forms, Y is always an array. 


FFT Command 


Purpose: 


Calculate the discrete Fourier transform (DFT) or inverse discrete Fourier 
transform (IDFT) of complex arrays. 
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1. FFT(XR,XI,Q,n,sign,ZR,ZI) 
2. FFT(X,Q,n,sign, ZR, ZI) 


Input: 
XR,XI 


Remarks: 


Arrays of length Q containing the real and imaginary parts of 
the complex numbers to be transformed. 


Array of length Q containing the data set to be transformed 
if it is real. 


Integer containing the length of the input array(s). 


Integer (<Q) containing the number of transformed values to 
store. 


Integer indicating whether the DFT or IDFT should be done 
(1 and —1 respectively). 


Arrays of length n containing the first n elements of the real 
and imaginary parts of the transform. 


1. The second form is the same as the first except that the imaginary part 
of the numbers to be transformed is assumed to be Zero. 


2. The FFT command will use the FFT algorithm if the largest prime fac- 
tor of Q is <23 and the product of the square-free prime factors is <210. 
Otherwise, FFT will use the defining formula for the calculation. If this 
happens, the time required to calculate the transform may be quite exten- 
sive. For example, on a PC with a 4.77-MHz clock, it takes approximately 
2.5 seconds to do a transform of length 101, about 10 seconds for Q=222" 
and 40 seconds for Q=444. 


3. Workspace=2Q if the fast Fourier transform can be used, and 2Q+2n if the 
defining formula is used. 
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FILT Command 


Purpose: 


Provide a linear filter of an array, that is, 


Pp 
Y(i) =) BjX(p+i-j), t=1,....n—p. 
j=0 


Format: 


Y=FILT(X, beta, beta0,n,p) 


Input: 

X - Array of length n containing the data to be filtered. 

n - Integer containing the length of the data to be filtered. 

beta - Array of length p containing coefficients of lags 1,..., p. 

betaO - Real scalar containing the coefficient for lag 0. 

Pp - Integer (<n) containing the length of the filter. 
Output: 

MM - Array of length n-p containing the result of the filter. 
Remark: 


1. Workspace=2n. 


GOTO Command 


Purpose: 


Provide unconditional branch for macros. 


Format: 


GOTO (label) 
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Input: 


label - The next line to be executed in the macro currently being 
executed is the line that has a ; in column 1 followed by the 
character string matching what is given by the argument of 
the GOTO command. 


eS SS ee eee ee 


GRMENU Command 


eee 


Purpose: 


To place a graphics menu onto a graphics screen. 


Format: 


GRMENU (hmin, hmax, vmin, vmax) 


Remarks: 


1. Issuing this command places the graphics menu on a graphics screen from 
which one can print or save the screen, or use the FIND utility. 


2. The arguments are the real-world values (as opposed to pixel coordinates) 
of the ends of the horizontal and vertical axes of a graph that is on the 
screen. Thus hmin must be less than hmax and vmin must be less than 
vmax. 


3. If there is more than one set of axes on a screen, the user can successively 
use the FIND utility on the different axes by (1) issuing the PLOTSIZE 
command to tell TIMESLAB which axes to look at, (2) issuing the GRMENU 
command with the endpoints of that set of axes, and (3) striking F1 to 
invoke the FIND utility. 


a ee 


GS Command 


a ee ee, 


Purpose: 


To find the QR decomposition of an (n x m) matrix using the modified 
Gram-Schmidt decomposition algorithm. 


Format: 


GS(X,n,m,Q,R, ier) 
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Input: 
X - Array of length n*m containing the matrix to be decomposed. 
n,m - Integers containing the dimensions of X. 
Output 
Q - An array of length n*m containing the orthogonal matrix in 
the decomposition. 
R - An array of length m*m containing the unit upper triangular 
matrix in the decomposition. 
ier - Integer variable indicating whether or not the matrix to be 
decomposed was judged to be singular (0 means no, 1 means 
yes). 
Remarks: 


1. The arrays Q and R are not formed if ier=1. 


2. Workspace=nm+m?. 


HELP Command 


Purpose: 


Provide an on-line manual. 


Format: 
1. HELP 
2. HELP (name) 
3. HELP (name, d) 


Remarks: 
1. The first form places a list of help topics on the screen. 


2. The second form places a description of the topic having name name on 
the screen. 


3. The third form is the same as the second except that the second argument 
is the letter of the device containing the file TSLABHLP (this must be A, 
SAOrts 


4. The HELP command is not allowed in the graphics mode. 
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HIST Command 


i Di, =... eee ee 


Purpose: 


Calculate and plot the histogram of a data set. 


Format: 


1. HIST(X) 
2. HIST(X,n) 


3. HIST(X,n,nbins) 


4. HIST(X,n,nbins,xmin, xmax, ymax) 


Input: 
X 


xmax 


ymax 


Remarks: 


Array of length n containing the data. 


Integer (4 0) whose absolute value is the number of points of 
the array X to use in finding the histogram. If n is positive 
(negative), then the bars will be solid (not solid). 


Integer (0 < nbins < 200) containing the number of intervals 
to use in the third and fourth forms. 


Real scalar containing the value to place at the left end of 
the horizontal axis in the fourth form. 


Real scalar containing the value to place at the right end of 
the horizontal axis in the fourth form. 


Real scalar containing the value to place at the top of the 
vertical axis in the fourth form. 


1. In the first two forms, HIST uses 


nbins = et] +1 


as the number of intervals, where rx and s x are the range and standard 
deviation of the data respectively. 


2. If ymax is entered as 0., hist will determine the scale on the vertical axis. 
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IF Command 


Purpose: 


Provide conditional and unconditional branches in a macro. See Section 
B.7 for examples of these branches. 


Format: 
1. IF(v1.exp.v2) 
2. IF(v1.exp.v2, label) 


3. IF(var,nn,nz,np) 


Remarks: 


1. In the first and second forms, vi and v2 are two scalar variables to be 
compared, while exp is one of lt, le, eq, ne, ge, or gt for less than, 
less than or equal, equal, not equal, greater than or equal, or greater than 
respectively. 


2. In the first form, if vi and v2 are related by the specified expression, then 
control of the macro is passed to the next line of the macro, while if they 
are not so related, then control passes to the line after an ENDIF command 
that matches the IF. 


3. In the second form, if vi and v2 are related by the specified expression, 
then control passes to the line in the macro that begins with a semicolon 
followed by a string of characters matched by what is given by the second 
argument of the IF. Otherwise, control passes to the line following the IF. 


4. In the third form, control of the macro is passed to the line having offset 
nn, nz, or np from the current line depending on whether the (integer or 
real) variable var is negative, zero, or positive. Blank lines and nondis- 
played comment lines are not counted when determining offsets. A positive 
(negative) offset means that control will pass downward (upward) in the 
file. 


INFO Command 


Purpose: 


Display a list of all variables that have been created in the current session. 
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Format: 


INFO 


Remarks: 


1. The number of and names of all arrays, reals, integers, and characters are 
given. 


2. The length and label of each array are given. 
3. The value of each real, integer, and character is given. 


4. The number of free elements out of the original 10,000 allowed for arrays 
is given. 


SS SE eee eee 


INFQNT Command 


SRY RESIST WEST oer ss we ee ees ee Le 


Purpose: 


Plot the informative quantile function for a data set. 


Format: 
1. INFQNT(X) 
2. INFQNT(X,n) 
a. INFQNT(X,n,xmed,xiqr) 
4. INFQNT(X,n,xmed,xigr,arg5) 


Input: 
x - Array of length n containing the data. 
n - Integer containing the length of the data set. 
arg5 - Having a fifth argument means that the median and in- 
terquartile range are returned without the plot being formed. 
Output: 
xmed - Real scalar variable containing the median of the data set. 
xiqr - Real scalar variable containing the interquartile range of the 


data set. 
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Remarks: 


1. An informative quantile plot is a plot of the sorted values of an array (af- 
ter subtracting the median and dividing by twice the interquartile range) 
versus (i — .5)/n,i=1,...,n. 


2. The true informative quantile plot of a uniform random variable is a 
straight line. This line is always put on the plot. 


INVPOLY Command 


Purpose: 

Find the first q coefficients of the reciprocal of the polynomial g(z) = 
1 a ped aj Zz. 
Format: 


beta=INVPOLY (alpha, p,q) 


Input: 
alpha - Array of length p containing the coefficients of the polynomial 
to be inverted. 
- Integer containing the length of alpha (>0). 
q - Integer containing the number of coefficients of the reciprocal 
to find (>0). 
Output: 
beta - Array of length q containing first q coefficients of the recip- 
rocal polynomial. 
Remark: 


1. Workspace=q. 


LABEL Command 


Purpose: 


Allow the user to change the label for an array or to place a string of 
characters on a graphics screen. 
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Format: 


AL, 


LABEL (X)=‘some character string’ 


2. LABEL (X)=<Y> 
3. LABEL(X)=<Y,Z> 
4, LABEL(col,row,iopth)=’some character string’ 
5. LABEL(name,offset)=’some character string’ 
Remarks: 
1. The first form gives the array X the label given by what is between the 


apostrophes. There can be at most 40 characters. 


. The second form replaces the current label for X with that of the array Y. 


. The third form takes the first 20 characters of each of the labels of the 


arrays Y and Z and forms a new label for X by concatenating them. 


. In the fourth form, the string of characters given between the apostrophes 


is placed on the screen starting at pixel location (col,row), where the 
lower left corner of the screen is location (0,0). The string is placed 
horizontally if iopth is 1 and vertically if iopth is 0. If placed horizontally, 
the lower left corner of the first character of the string is at (col,row), 
while if placed vertically, the upper left corner of the first character is at 
col,row). 


. In the fifth form, the string of characters given between the apostrophes is 


placed in a position relative to the set of axes specified by the most recent 
call to the PLOTSIZE command (or to the default axes if PLOTSIZE has 
not been called) according to the values given for the two arguments. The 
first argument can be x, y, or cap (in upper or lowercase) which mean that 
the string is placed relative to the horizontal axis, the vertical axis, or the 
caption position respectively. The second argument is an integer specifying 
how far from the closest possible position the string is to be placed. An 
offset of 0 means that the string is placed as close as possible (a caption 
would be overwritten), while an offset of 1 means the string will be placed 
eight pixels away from the closest position. This would be down, left, and 
up for x, y, and cap respectively. The offset can also be negative which 
puts the string in the opposite direction. Strings placed on the horizontal 
or vertical axis are centered on the axis, while those placed at the caption 
are placed in line with captions generated by graphics commands. 
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6. In the first, fourth, and fifth forms, variable integers or reals can be in- 
serted into a label by placing them within #’s (integers) or @’s (reals), for 
example, 
LABEL (X)=’ AR(#nord#) Spectra, RVAR=@rvarQ’ 
is the same as 


LABEL (X)=’AR(5) Spectra, RVAR=2.95’ 


if nord=5, rvar=2.95. 


LENGTH Command 


Purpose: 


Find the number of elements in an array. 


Format: 
1. LENGTH (X) 
2. LENGTH(X,n) 


Remark: 


1. The first form merely shows the length of X on the screen while the second 
form also places the length in the integer variable n. 


LINE Command 


Purpose: 


Form an array consisting of equally spaced real numbers. 


Format: 
1. X=LINE(n,a,b) 
2. X=LINE(n,a,b,arg4) 


Remarks: 


1. The first form finds X(i)=a+bi for i=1,...,n. 


534 


2: 


4. 
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In the second form, the fourth argument is used only as a signal that the 
second form is to be used and X is an array containing the n equally spaced 
reals between a and b inclusive. 


. This command is very useful in forming arrays of constants (first form 


with b=0), indices of time series (second form with a being the first time 
minus the sampling interval and b being the sampling interval), or finding 
the values at which to evaluate a distribution or polynomial (e.g., to get 
—1,—.99,...,.99,1, one can use the second form with n=201, a=-1, and 
b=1). 


Workspace=n. 


SS 


LIST Command 


MME Eo... eee 


Purpose: 


Display the values of one or more variables or direct the display to a file 


(including the printer). 


Format: 


i 
2. 


3. 
4. 
5. 


LIST(list of scalar variables) 
LIST (X) 

TES ECOG (0) 

LIST(X,n) 

LIST(X,n,ng, FORMAT) 


Remarks: 


ile 


The first form displays the values of the (real, integer, or character) vari- 
ables in the argument list. 


- The second form lists all of the elements of the array X, five per line in 


5G14.6 format. 


3. The third form lists only the label of the array X. 


- The fourth form lists the first n elements of the array X in 5G14.6 format. 


. The fifth form lists the first n elements of the array X, ng elements per line, 


in the format specified by FORMAT, which is a Fortran format statement 
without its parentheses, for example, LIST(X,100,10,10F7.2). 
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6. In any of the above forms, putting another argument which is a file name 
between apostrophes sends the display to that file instead of to the screen. 
This includes the printer (which has file name prn or 1pti in either upper 
or lowercase). 


7. The format 5G14.6 means that there are five numbers per line, with six 
digits used for each number unless the number is less than .01 or greater 
than 10°, in which case scientific notation is used. 


8. When listing an array, the index of the first element of each row in the 
display is listed also. 


9. Only one array can be displayed per command and scalars and arrays 
cannot be listed in the same command. 


LISTM Command 


Purpose: 


Display a matrix or three-dimensional array. 


Format: 

LISTM(A,n,m[,k] ) 

Input: 

A - Array to be listed. If considered a matrix, then its first n 
elements are its first column, and so on. If considered a three- 
dimensional array, its first n elements are the first column of 
the first matrix, the next n are the second column, and so on. 

n - Integer containing the number of rows of the matrix(ces). 

m - Integer containing the number of columns of the matrix(ces). 

- Integer containing the number of matrices. 
Remark: 


1. The display is done in 6F 12.6 format. 
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annem ee ee ee ee ee es ee 
LISTSP Command 


a a 


Purpose: 


Display the elements of a spectral array along with their frequencies. 


Format: 


LISTSP(£,Q) 


Input: 
f - Array of length m =[Q/2]+1 containing some spectral quan- 
tity at the frequencies (j — 1)/Q, j = 1,...,m. 
Q - Integer containing the number of frequencies at which the 


spectral quantity was calculated. 


re ee 
LOGE Command 


ee 


Purpose: 


Calculate the natural log of a real scalar or of an array. 


Format: 
1. y=LOGE (x) The input x and the output y are real scalars. 


2. Y=LOGE(X,n) The input X is an array of length at least n, and the 
output Y is an array of length n containing the natural log of the elements 
of X. 


Remarks: 


1. If any of the input reals are nonpositive, an error message is displayed and 
the output is not found. 


2. Workspace=M. 
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MACORR Command 


Purpose: 


Find the variance R(0) and correlations p(1),..., p(M) of an MA(q, B, 07) 
process. 


Format: 


rho=MACORR(beta,q,rvar,M,RO) 


Input: 
beta - Array of length q containing coefficients B. 
q - Integer containing order q (>0). 
rvar - Real scalar containing error variance 0? (>0). 
M - Integer containing the number of autocorrelations to find 
(>0). 
Output: 
RO - Real scalar variable containing the variance of the MA pro- 
cess. 
rho - Array of length M containing the MA autocorrelations. 
Remark: 


1. Workspace=M. 


MACRO Command 


Purpose: 


Invoke or reinvoke a macro. 


Format: 
1. MACRO(fname) 
2. MACRO(fname, label) 
3. MACRO 
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Remarks: 
1. The first form invokes the macro in the file having name fname. 


2. The second form invokes the macro in the file having name fname at the 
line in the file having (1) a semicolon in column 1, (2) then a string of 
characters matching that given as the second argument. 


3. The third form reinvokes the most recently interrupted macro. 


4. See Section B.7 for more information about macros. 


ee ee ee 
MADT Command 


Se eee 


Purpose: 


Generate a realization from a Gaussian MA(q, B, 0?) process. 


Format: 


X=MADT (beta,q,rvar,seed,n) 


Input: 

beta - Array of length q containing coefficients B. 

q - Integer containing order q (>0). 

rvar - Real scalar containing error variance o? (>0). 

seed - Real scalar containing the seed for the random number gen- 

erator. 

n - Integer (>q) containing the number of observations desired. 
Output: 

X - Array of length n containing the realization. 
Remark: 


1. Workspace=n. 


ee ae 
MASP Command 


a 


Purpose: 


Calculate the spectral density of an MA(q, 8,0”) process. 
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Format: 


f=MASP (beta,q,rvar,Q) 


Input: 

beta - Array of length q containing coefficients . 

q - Integer containing order g (>0). 

rvar - Real scalar containing error variance a? (>0). 

Q - Integer (>q) containing the number of frequencies between 0 
and 1 at which to calculate the spectral density. The greatest 
prime factor of Q can be at most 23, and the product of its 
square-free prime factors can be at most 210. 

Output: 

f - Array of length m =[Q/2]+1 containing MA spectra at fre- 

quencies (j — 1)/Q,7 =1,...,m. 
Remark: 


1. Workspace=2Q. 


MAXMIN Command 


Purpose: 


Find the values and indices of the maximum and minimum of an array. 


Format: 


MAXMIN(X,n,xmax, imax ,xmin, imin) 


Input: 
X - Array of length n for which to find max and min. 
n - Integer containing the length of X. 
Output: 
xmax - Real scalar containing the largest value in the array X. 
xmin - Real scalar containing the smallest value in the array X. 


imax - Integer containing the index in X of xmax. 
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imin - Integer containing the index in X of xmin. 


eee 
MCHOL Command 


a eee 


Purpose: 


To find the modified Cholesky decomposition of a symmetric, positive 
definite matrix. 


Format: 
MCHOL(A,n,L,D,ier) 
Input: 
A - Array of length n*n containing the matrix to be decomposed. 
n - Integer containing the size of the square matrix to be decom- 
posed. 
Output: 
ib, - Array of length n*n containing the unit lower triangular factor 
in the decomposition. 
D - Array of length n*n containing the diagonal factor in the 
decomposition. 
ier - Integer indicating whether or not A was judged to be posi- 
tive definite (0 means positive definite, 1 means not positive 
definite). 
Remarks: 


1. If ier=1, then L and D are not formed. 


2. Workspace=2n2+n. 


Beer Tremreeenernr gc er ee ee 
MDEL Command 


a re EE ee ee 


Purpose: 


To delete specified rows of a matrix. 
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Format: 
B=MDEL(A,n,m,ri,...,rk) 
Input: 
A - Array of length n*m containing the matrix whose rows are to 
be deleted. 
n,m - Integers containing the size of the matrix. 
ri,... - Integers containing the numbers of the rows to be deleted. 
Output: 
B - Array containing the (n-k)xm output matrix. 
Remark: 


1. Workspace= (n-k)m. 


MINV Command 


Purpose: 


To find the inverse of a matrix. 


Format: 


B=MINV(A,n,ier) 


Input: 
A - Array of length n*n containing the matrix to be inverted. 
n - Integer containing the dimension of A. 
Output: 
ier - Integer indicating whether or not A was judged nonsingular 
(0 means nonsingular, 1 means singular). 
B - Array of length n*n containing the inverse of A. 
Remarks: 


1. If ier=1, then B is not formed. 


2. Workspace=2n?. 
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eee 


MMULT Command 


ee eee ee 


Purpose: 


To multiply matrices. 


Format: 
1. C=MMULT(A,B,n,m,k) 
2. C=MMULT(A,B,n,m) 
3. C=MMULT(A,B,n) 


Remarks: 
1. In the first form, A is nxn, B is mxk, and C=AB. 
2. In the second form, A is nxm, B is nxm, and C=A’B. 
3. In the third form, A and B are nxn and C=AB. 


4. The workspaces for the three forms are nk+nm, m’, and 2n? respectively. 


ee eee 


MULTPOLY Command 


a eee 


Purpose: 


Find the coefficients of the product of the two polynomials 


P q 
g(z) =14+ ie ajzi and h(z)=1+4 Ms Bz". 
j=1 


j=l 


Format: 


gama=MULTPOLY (alpha, beta, P>q) 


Input: 
alpha - Array of length p containing the coefficients of the first poly- 
nomial. 
beta - Array of length q containing the coefficients of the second 
polynomial. 


Pp - Integer containing the degree of the first polynomial (p>0). 
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q - Integer containing the degree of the second polynomial (q>0). 
Output: 
gama - Array of length p+q containing the coefficients of product 
polynomial. 


NOTES Command 


Purpose: 
To allow users to create and access their own help files. User-defined help 
files are called notes. 
Format: 
1. NOTES (fname) 


2. NOTES (fname, notename) 


Remarks: 


1. A note consists of a series of lines in an ASCII file. The first line of a 
note consists of the name of the note starting in column 1. The rest of the 
note is a series of lines (column 1 must be blank in each line) containing 
information about something. 


2. The first form results in a list of the note names on the file having name 
fname. 


3. The second form results in the display of the note having name notename 
in the file called fname. 


OVEROFF Command 


Purpose: 

Tell TIMESLAB to henceforth ask for permission to replace any previously 
defined variable whenever a command would result in such a replacement. 
Format: 


OVEROFF 
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rr ee tt 


OVERON Command 


ee 


Purpose: 

Tell TIMESLAB to henceforth automatically overwrite any previously de- 
fined variable whenever a command would result in such a replacement. 
Format: 


OVERON 


Remarks: 
1. The BATCHON command also causes automatic overwriting. 
2. OVERON is cancelled by OVEROFF. 
3. When TIMESLAB starts, it is in the OVERON mode. 


eee 


PAGE Command 


MERITS G Geey oo ee ee eee 


Purpose: 


Position the printer to the top of the next page. On a nonlaser printer 
this is where the print head is on the paper when the printer is turned on or 
the PC is reset. On a laser printer, PAGE causes a page eject. 


Format: 


PAGE 


EEE. eee 


PARCORR Command 


a ee 


Purpose: 


Calculate partial autocorrelations and optionally residual variances for a 
data set. 


Format: 


part=PARCORR(X,n,M[,rvar]) 
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Input: 
X - Array of length n containing the data. 
n - Integer containing the length of X. 
M - Integer (0 < M < n) containing the number of partial auto- 
correlations to find. 
Output 
part - Array of length M containing partial autocorrelations. 
rvar - Ifthis argument is present, it is an array of length M containing 
standardized residual variances. 
Remark: 


1. Workspace=2n+4M. 


PARTAR Command 


Purpose: 


Calculate the coefficients of an AR(p, a, 0”) process given its partial auto- 
correlations. 


Format: 


alpha=PARTAR (part, p) 


Input: 
part - Array of length p containing partial autocorrelations. 
p - Integer containing order p (>0). 
Output: 
alpha - Array of length p containing coefficients a. 
PAUSE Command 
Purpose: 


Provide a means of obtaining user input during execution of a macro or 
else just have a macro pause. 
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Format: 


PAUSE 


Remarks: 


1 


. When TIMESLAB encounters a PAUSE command during execution of a 
macro it issues the message “Strike F9 to break MACRO, anything else to 
go on.” Striking F9 at this point will cause the macro to stop executing 
and TIMESLAB will issue its usual prompt. Then the user can issue other 
commands and then restart the macro by entering the command MACRO. 


. The writer of the macro can precede the PAUSE command by some echoed 
comments, that is, comment lines that begin with two semicolons describ- 
ing what variables need to be defined. 


SSS ee 


PLOT Command 


a 


Purpose: 


General purpose Y versus X plot. 


Format: 


1 


2 
3 
4. 
5 
6 


. PLOT(X) 

. PLOT(X,n) 

. PLOT(X,n,xmin, xmax) 

PLOT (X,n,hmin, hmax, vmin, vmax) 
-1PEQT (X,Y sn) 

. PLOT(X,Y,n,xmin,xmax, ymin, ymax) 


Remarks: 


1 


. The first form graphs the elements of the input array X versus their index 
with PLOT determining the scale of the plot so that the ends of the axes 
correspond to the minimim and maximum of X on the vertical axis, the 
number 1 on the left of the horizontal axis, and the number of points being 
plotted put at the right end. The entire array X is plotted. 


. The second form is the same as the first except that the first n elements 
of the array X are plotted. 
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3. 


The third form is the same as the second except that the user specifies 
xmin and xmax, that is, the values to be used at the ends of the vertical 
axis. 


. The fourth form again plots the first n elements of X versus their index 


except now the values at the ends of both axes are specified by the user 
in hmin, hmax, vmin, vmax. 


. In the fifth form, an array Y is plotted on the vertical axis, an array X is 


plotted on the horizontal axis, and the values at the ends of the axes are 
the actual minima and maxima of the arrays. 


. The sixth form is the same as the fifth except that the values at the 


ends of the axes are specified by the user in the real scalar input values 
xmin,xmax, ymin, ymax. 


. In any of the forms, if n is negative then a point plot is produced. This 


allows a scatterplot to be produced. 


a ee eee ae 


PLOT2 Command 


ee ae ee ee 


Purpose: 


Superimpose the graphs of two arrays. 


Format: 


il 
2 
3. 
4, 


PLOT2(Y1,Y2,ny1,ny2,type1,type2) 
PLOT2(Y1,Y2,ny1,ny2,type1,type2,X) 
PLOT2(Y1,Y2,ny1,ny2,type1 , type2,xmin,xmax, ymin, ymax) 
PLOT2(Y1,Y2,ny1,ny2,type1 , type2,X,xmin,xmax, ymin, ymax) 


Input: 


Ye - Array of length ny1. 
Y2 - Array of length ny2. 
nyi,ny2 - Integers containing the lengths of Y1 and Y2. 


typel - Integer indicating how Y1 is to be plotted. The following 
values are available: 


1 Line plot. 
2 Point plot. 
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xmax 


Remark: 
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3 If both type1 and type? are 3, then an x is placed at 
the ith points of both plots and vertical lines connect 
them. 


4 Line plot with x’s at each point. 
Same as type1 except for Y2. 


In the second and fourth forms, an altay containing values 
for the horizontal axis. 


In the third and fourth forms, a real scalar containing the 
value to be used at the left end of the horizontal axis. 


In the third and fourth forms, a real scalar containing the 
value to be used at the right end of the horizontal axis. 


In the third and fourth forms, a real scalar containing the 
value to be used at the bottom of the vertical axis. 


In the third and fourth forms, a real scalar containing the 
value to be used at the top of the vertical axis. 


1. In the first and third forms, the values on the horizontal axis are the 
numbers 1,2,.... 


a ee 


PLOTCSP Command 


SSS SSS eee eee 


Purpose: 


To plot a spectral array on a nonlog scale. 


Format: 


PLOTCSP (£ ,Q) 


Remarks: 


1. f is an array of length [Q/2]+1. 
2. £(i) versus (i-1)/Q, i=1,...,[Q/2]+1, is plotted. 
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PLOTK Command 


Purpose: 


To allow the superimposing of any number of plots on the same axes. 


Format: 


PLOTK(X,Y,n,k,type[,xmin, xmax, ymin, ymax] ) 


Input: 
k - Integer containing the number of plots to be superimposed. 
X,Y - Arrays containing the arrays to be plotted. 


n - In general, an array of length k determining the number of 
points in each plot. The first n(1) elements of X and Y are 
plotted versus each other, the next n(2) elements are plot- 
ted versus each other, and so on. If each plot has the same 
number of points, then the common number can be inputted 
as an integer n. 


type - Array of length k containing the type of plot to be produced 
for each plot. A value of 2 gives a line plot, 10, 11, 12, 13, 
and 14 give a line plot with symbols at each point (10 gives a 
single pixel, 11 gives an x, 12 gives a square, 13 gives a lower 
triangle, and 14 gives an upper triangle), while 30-34 give 
just point plots with 30 being a single pixel, 31 being an x, 
32 a square, 33 a lower triangle, and 34 an upper triangle. If 
k is 1, then type can be a scalar. 


Remarks: 


1. If the last four arguments are included, they are real scalars specifying the 
endpoints of the axes. 


2. The labels for Y and X are placed above the completed screen. 


PLOTOFF Command 


Purpose: 


To switch the screen from the graphics mode to the text mode. 
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Format: 


PLOTOFF 


Remark: 


1. If the screen is already in the text mode, then issuing this command will 
clear the screen. 


ee ee ee 


PLOTON Command 


Sn ec arr ce Os a 


Purpose: 


To switch the screen from the text mode to the graphics mode. 


Format: 


PLOTON 


Remarks: 


1. After this command is issued, the prompt will stay at the top of the 
screen so that the user can issue a series of graphics commands without 
the prompt and output of commands interfering with what is on the rest 
of the screen. 


2. If the screen is already in the graphics mode, then this command will clear 
the screen. 


Taunt Saab ae Se ee 


PLOTSIZE Command 


Sueur oe ee eee 


Purpose: 

To specify various parameters to be used in graphics commands that are 
issued in the future. 
Format: 


ks PLOTSIZE (npixx npixy ,nx1,ny1,nticsx,nticsy ,nclabx, 
ndecx ,nclaby ,ndecy) 


Pa PLOTSIZE (npixx,npixy ,nx1,ny1,nticsx,nticsy) 
os PLOTSIZE (npixx ,npixy ,nx1,ny1) 
4. PLOTSIZE(kode) 


nyl 

nticsx 
nticsy 
nclabx 


ndecx 


nclaby 


Remarks: 
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The length in pixels of the horizontal axis. 

The length in pixels of the vertical axis. 

The column number where the origin is to be. 
The row number where the origin is to be. 

The number of tic marks on the horizontal axis. 
The number of tic marks on the vertical axis. 


The number of characters to be allowed for the numerical 
labels on the tic marks on the horizontal axis. This includes 
a decimal point and a minus sign if there is one. 


The number of places to the right of the decimal point in the 
tic mark labels on the horizontal axis. 


The number of characters to be allowed for the numerical 
labels on the tic marks on the vertical axis. This includes a 
decimal point and a minus sign if there is one. 


The number of places to the right of the decimal point in the 
tic mark labels on the vertical axis. 


The fourth form is equivalent to the third form with the 
following values of kode using the indicated values of npixx, 
npixy, nxi, nyl: 


0 480120 55 20 (default values) 

260 60 50 120 (upper left quarter of the screen) 
260 60 50 20 (lower left quarter of the screen) 
260 60 50 20 (upper right quarter of the screen) 
260 60 50 20 (lower right quarter of the screen) 
260 120 50 20 (left half of the screen) 

260 120 360 20 (right half of the screen) 

480 60 50 110 (upper half of the screen) 

480 60 50 20 (lower half of the screen) 


ono a -& &© WS FF 


1. When TIMESLAB is started, the default values of the arguments are: 
npixx:480, npixy:120, nx1:55, ny1:30, nticsx:10, nticsy:10, nclabx:8, 
ndecx:2, nclaby:6, ndecy:2. 
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2. The value of npixx+nx1+8*nclaby+5 should be less than 640 for the hor- 
izontal axis to fit on the screen, while the value of npixy+ny1+12 should 
be less than 199 for the vertical axis to fit. 


3. A value of zero for nclabx or nclaby results in there being no tic mark 
labels on the horizontal or vertical axis respectively. 


ee ee ee ee eS 
PLOTSP Command 


[Ts -... e 


Purpose: 


Plot a spectral array or several spectral arrays on a log scale. 


Format: 
, PLOTSP (f ,Q,d) 
2 PEOTSE Gills Qed uals )eaee li!) 


Remarks: 


1. The first form produces a plot of 


log(£(i)/d) versus (i —1)/Q, 
for i = 1,...,[Q/2] +1. 


2. The second form superimposes several such plots. Note that Q must be the 
same for each plot while the divisors di, ...,dk can be different. TIMES- 
LAB knows how many plots to superimpose because there are three argu- 
ments for each one. 


3. The label (or labels) for f (or f1,... ,fk) are placed above the completed 
screen. 


a a 
POLAR Command 


SSS eee ee 


Purpose: 


Find amplitudes and phases of complex numbers. 


Format: 
1. POLAR(xr,xi,amp,phase) 
2, POLAR(xr,xi,n,amp, phase) 
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Input: 
xr,xi_- In the first form, real scalars containing the real and imag- 
inary parts of a complex number, and in the second form 
arrays of length at least n containing the real and imaginary 
parts of a set of complex numbers. 
n - Integer containing the length of xr and xi (>0). 
Output: 
amp - Real scalar (first form) or an array (second form) of length n 
containing the amplitudes. 
phase - Real scalar (first form) or an array of length n (form 2) con- 
taining the phases. 
Remarks: 


1. See Problem T1.2 for a discussion of how the amplitude and phase of a 
complex number are defined. 


2. Workspace=2n in the second form. 


POLY Command 


Purpose: 


Evaluate a polynomial }7%_, ajx at a specified set of values. 


Format: 
1. Y=POLY(alpha,p,n,a,b) 
2. Y=POLY(alpha,p,n,X) 


Input: 
alpha - Array of length p+1 containing the coefficients of the polyno- 
mial of degree p (alpha(i) is the coefficient of power i-1). 
- Integer containing the degree of the polynomial (>0). 
n - Integer containing the number of points at which to evaluate 
the polynomial (>0). 
a,b - In the first form, input scalars telling POLY to evaluate the 


polynomial at the n equally spaced values between a and b 
inclusive. 
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x - In the second form, an array of length n containing the points 
at which to evaluate the polynomial. 


Y - Ifn=1, b is ignored and Y is a real scalar variable containing 
the desired value of the polynomial. If n>1, then Y is an 
array of length n containing the values of the polynomial. 


ee eee ee 


POLYROOTS Command 


a ee 


Purpose: 


To find the zeros of the polynomial 
Pp . 
g(z) = >> Qyzae 
j=0 


Format: 


POLYROOTS (alpha, alpha0, p,Maxit,rr,ri,ier) 


Input: 
alpha - Array of length p containing ay through ap. 
alphaO - Real scalar containing ag. 
Pp - Integer containing the degree of the polynomial (>0). 
maxit - Integer (1 < maxit < 1000) containing the maximum number 
of iterations in the procedure. 
Output: 
ave - Array of length p containing the real parts of the roots. 
ri - Array of length p containing the imaginary parts of the roots. 
ier - Integer variable indicating whether or not the procedure con- 
verged (0 means yes, 1 means no). 
Remarks: 


1. If ier=1, then rr and ri are not formed. 


2. Workspace=9(p+1). 
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PRINT Command 


Purpose: 


Print an array or send its display to a file. 


Format: 


The PRINT command is identical to the form of the LIST command that 


directs the output to the printer except that the argument specifying the file 
name is not used. 


PRINTER Command 


Purpose: 


Set printer characteristics. 


Format: 


PRINTER(codei,...,codek) 


Remarks: 
1. The arguments are all input integers. 


2. If code is not 27, then the arguments have the following meanings for an 
IBM Graphics Printer (or compatible): 
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Options in PRINTER Command 


IOPT Meaning IOPT Meaning 
1 Compressed print 2 Compressed print off 
3 Use character set 1 4 Use character set 2 
5 Emphasized print on 6 Emphasized print off 
i Double strike on 8 Double strike off 
9 Wide size print on 10 Wide size print off 


30+n Advance n lines 
DE MeRuneEee oe ae 


3. If iopt1=27, then the rest of the arguments are sent directly to the printer. 
The user’s printer manual should be consulted to find what codes are 
useful. An example is PRINTER (27 ,27,38,108,49,79) which puts the 
H-P Laserjet into the landscape mode. 


4. If a printer other than the IBM Graphics Printer is being used, then the 
PRINTSEL command should be used to tell TIMESLAB what the printer 
is. 


ee 


PRINTSEL Command 


et 


Purpose: 

To select which of three possible types of printers is attached to the first 
parallel port. 
Format: 


PRINTSEL (n) 


Input: 


n - Integer specifying which printer type is being used. The fol- 
lowing choices are available: 


1 IBM Graphics Printer (or compatible). 
2 HP Laserjet Printer (or compatible). 
3 Toshiba 321 (or compatible). 
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PROMPTOFF Command 


Purpose: 


Henceforth the lines of a macro are not displayed on the screen as they 
are issued. 


Format: 


PROMPTOFF 


PROMPTON Command 


Purpose: 


Cancel the effect of PROMPTOFF. 


Format: 


PROMPTON 


PSOFF Command 


Purpose: 


Cancel the effect of PSON. 


Format: 


PSOFF 


PSON Command 


Purpose: 

Henceforth any plot that goes to the screen will also be dumped to the 
printer until PSOFF is issued. 
Format: 


PSON 
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Remark: 


1. Screen dumps can be printed on either an IBM Graphics Printer (or com- 
patible), an HP Laserjet Printer (or compatible), or a Toshiba 321 (or 
compatible). See the PRINTSEL command and Section B.5.4 for more in- 
formation. 


ee ere es 


QTEST Command 


SSS eee 


Purpose: 


Perform the modified portmanteau test for white noise. 


Format: 


pval=QTEST(rho,M,p,q,n,Q) 


Input: 
rho - Array of length M containing autocorrelations of the data 
being tested. 
M - Integer containing the length of rho. 
P.q - If rho are autocorrelations from an ARMA(p, g) model, then 
p and q are integers containing the orders. Otherwise, they 
should be given the value zero. M-p-q must be positive. 
n - Integer (> M) containing the sample size for the data being 
tested. 
Output 
Q - Real scalar variable containing the test statistic. 
pval - Real scalar variable containing the p-value of the test. 


SS St 


QUIT Command 


SS ee 


Purpose: 


Quit execution of TIMESLAB and return to DOS. 


Format: 


QUIT 
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READ Command 


Purpose: 


Read a data set from a disk file. 


Format: 


READ (fname, X,n[,nskip)] 


Input: 
fname - Name of the file containing the data. 
nskip - If this argument is present, it is an integer containing the 
number of data sets to skip on the file before reading the set 
that is desired. If nskip is not present, then the first data 
set is read. 
Output: 
X - Array of length n containing the data. 
n - Integer variable containing the length of X. 
Remark: 


1. See Section B.4.8 for more information about data files. 


RECORD Command 


Purpose: 
Keep a record on a file of the output displayed by several of the TIMES- 
LAB commands. 
Format: 
1. RECORD (fname) 
2. RECORD (close) 
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Remarks: 


1. The first form begins the recording and tells TIMESLAB to use the file 
having name fname. The second form tells TIMESLAB to end the record- 
ing and to close the file. If a file already exists having name fname, RECORD 
will append the new material onto the end of the existing file. If the device 
being used for recording becomes full, RECORD will stop adding material to 
the file without necessarily telling the user. 


2. Any command entered at the keyboard will be recorded as well as informa- 
tion displayed by the stepwise form of the REG command and information 
displayed by the ARMASEL, DTAR, INFO, LIST, LISTM, and SEASEST com. 
mands. 


MEMES ees a 


REG Command 


le Ga Se 


Purpose: 


Regress the array Y on the (n x m) matrix X. 


Format: 
1. REG(Y,X,n,m,beta,RSS,t,res) 
Ds REG(Y,X,n,m,pval,kopt ,k,beta, ind,RSS) 
ak REG(Y,X,n,m,beta, VIBETA,RSS,ni,t,res, Irssio) 


Remarks: 


1. The first form is referred to as the simple form, the second as the stepwise 
form, and the third as the general form (see Section A.5). 


2. The workspaces required are: 


2(n+m) +nm+(m+1)2 for form 1 
(m+1)n+(m+1)?+3m for form 2 
(m+1)?+m+n for form 3 and n>m 


(n+m) ?+m+n for form 3 and n<m. 
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REPLACE Command 


Purpose: 


To replace selected elements of an array. 


Format: 
1. Y=REPLACE(X,X1I,R,nr) 
2. Y=REPLACE(X,n, iopt,a[,b,c]) 


Remarks: 


1. In the first form, the output array Y is the same as the input array X except 
that the elements of X having indices contained in the array XI (which is of 
length nr) are replaced by the corresponding elements of the array R. Thus 
the element of X having index XI[i] is replaced by R[il, for i=1,...,nr. 
If all of the replacing values are the same, then R can be entered as a real 
scalar. 


2. The second form is used to do various kinds of truncations of the first 
n elements of the input array X. The following options are available as 
specified by the input integer iopt and how many arguments there are: 


1 Values of X that are less than a are replaced by a (4 
arguments). 


2 Values of X that are greater than a are replaced by a 
(4 arguments). 


3 Values of X that are less than a are replaced by a, 
while values of X that are greater than b are replaced 
by b (5 arguments). 

4 Values of X between a and b (inclusive) are replaced 
by c (6 arguments). 


5 Values of X that are less than or equal to a are re- 
placed by b, while values of X that are greater than 
a are replaced by c (6 arguments). 
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RESCREEN Command 


oS a eee eS ee ee 


Purpose: 


To display a previously created graphics screen or screens. 


Format: 
1. RESCREEN (fname) 
2. RESCREEN(fn1,... ,fnk,iopt) 
3. RESCREEN(fn, j,k, iopt) 


Remarks: 
1. The first form recreates the screen contained in the file called fname. 


2. The second form recreates successively screens in files fni,... ,fnk either 
erasing in between (iopt=1) or superimposing them all (iopt=2). 


3. The third form is similar to the second form except that file names all start 
with the characters specified by fn (which can be any string of characters) 
and end with the integers j,j+1,...,k. Further, iopt must be either 3 
or 4 which has the same meaning as 1 or 2 in the second form. 


4. In the second and third forms, adding 10 to iopt causes a pause between 
screens until a carriage return is struck. 


ee 
RESTART Command 


See a a es eee 


Purpose: 


To restart TIMESLAB. 


Format: 


RESTART 


Remark: 


1. When TIMESLAB sees this command it begins execution at its own first 
line. Thus the welcoming screen reappears and everything is reinitialized. 


SEC C.2 DETAILED DESCRIPTION OF COMMANDS 563 


REVERSE Command 


Purpose: 
To create an array which is the same as another with its elements reversed. 


Format: 


Y=REVERSE (X ,n) 


Input: 

ey - Array of length n. 

n - Integer containing length of X. 
Output: 

VW - Array of length n containing the elements of X in reverse 

order. 
ROOTSPOLY Command 

Purpose: 


To find the coefficients of a polynomial 


P 
g(z) =1+ ye azz) 


j=1 
given its zeros. 
Format: 
ROOTSPOLY (rr,ri,p,alpha) 
Input: 
rr,ri_ - Arrays of length p containing the real and imaginary parts of 
the zeros of the polynomial. 
Pp - Integer containing the degree of polynomial. 
Output: 


alpha - Array of length p containing coefficients. 
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Remarks: 


1. In the POLYROOTS command, we allowed the coefficient of 2° to be different 
than 1. Thus if the commands POLYROOTS and ROOTSPOLY are issued suc- 
cessively, the coefficients from ROOTSPOLY will be the original coefficients 
divided by the original constant term. 


2. Workspace=9(p+1). 


a 


SAVE Command 


re ae 


Purpose: 


To save an array onto a disk file. 


Format: 


SAVE(X,n, fname) 


Input: 
X - Array of length n to be saved. 
n - Integer containing the length of X. 
fname - Name of file where array will be saved. 
Remarks: 


1. The array is saved in 5E14.6 format. 


2. If the disk file already exists, then this data set will be appended to the 
file. 


a ee OU ay 


SAVESC Command 


ra 


Purpose: 


To save a compressed image of the next graphics screen onto a file. 


Format: 


SAVESC (fname) 


SeciC:2 


Remarks: 
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1. If this command is issued in the text mode, then the next time that a graph 
is formed using one of the HIST, INFQNT, PLOT, PLOTCSP, PLOTK, PLOTSP, 
or PLOT2 commands, the screen will be saved onto a disk file whose name 
is given as the argument of the SAVESC command. 


2. If this command is issued in the graphics mode, then whatever is on the 
screen (with the top eight rows of pixels erased) will be saved to the 


specified file. 


SEASEST Command 


Purpose: 


Fit a multiplicative subset ARMA model to data X of length n. 


Format: 


SEASEST(X,n,ords,coeffs,lags,it,eps,nb,rvar,ier,sds[,e]) 


Input: 
X,n - 


ords - 


lags - 


coeffs - 


it - 


eps - 


Data and length of data. 


An array of length 5 containing the full and subset AR orders, 
followed by the full and subset MA orders, followed by a 1 if 
a constant term is in the model or a 0 if it is not. 


An array containing the lags (if any) in the model. If both 
the subset AR and MA orders are zero, no array called lags 
need be formed, but an argument must be included. 


An array containing starting values for the coefficients that 
are included in the model in the order full AR, subset AR, 
full MA, subset MA, and the mean of X. 


An integer containing the number of iterations to allow in the 
estimation procedure. If it is negative, then -it iterations 
are allowed and the values of the coefficients for the successive 
iterations are displayed on the screen. If it is 1, then SEASEST 
only evaluates rvar and sds. 


A real scalar containing a convergence criterion. If the max- 
imum value of successive iterates differs by less than eps, 
SEASEST judges that the algorithm has converged. 
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nb 


Output: 


coeffs 


rvar 


ier 


sds 


Remarks: 
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An integer containing the number of back forecasts to use in 
determining initial values in the recursion used in evaluating 
the sum of squares of residuals function (> 0). 


An array containing the final values reached for the param- 
eters in the iterative process. coeffs is not changed from 
input if it=1. 


Estimate of error variance. 


An integer variable indicating whether or not convergence was 
achieved (0 means yes, 1 means no), if a singular matrix was 
encountered (2), or whether the algorithm could not continue 
even though convergence was not reached (3 or 4). If this final 
alternative happens, different starting values or convergence 
criteria may lead to convergence. 


An array containing the standard errors of the estimates. 


An optional argument which if present is an array of length n 
containing the one step ahead prediction errors corrrespond- 
ing to the n values of X. 


1. In the notation of Chapter 3, we have 


ords=<p,P,q,Q,M> 


2. Workspace=3 (n+nb) +5K+3maxp+maxq+K (n+nb) +2(K+1)?, where K is the 
sum of the elements of ords, and maxp and maxq are the largest AR and 
MA lags in the model when it is expressed as an ARMA model. 


ee 


SEASPRED Command 


RNY a rr 


Purpose: 


Predict values of a multiplicative subset ARIMA process given data X of 


length n. 


Format: 


SEASPRED (X,n,ords,coeffs, lags, rvar,tf,tl,hl,conf, 


xp,xpl,xpu, ier) 


SECHC:2 


Input: 
X,n 


ords 


rvar 


tip te. 


hl 


conf 


Output: 
xp 


xpl 
xpu 


ier 
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The data set and its length. 


An array of length 8 containing the full and subset. AR orders, 
followed by the full and subset MA orders, followed by a 1 if 
a constant term is in the model or a 0 if it is not, followed 
by the number of first differences in the model, the number 
of Sth differences in the model, and finally the value of S. 


An array containing the lags (if any) in the model. If both 
the subset AR, and MA orders are zero, no array called lags 
need be formed, but an argument must be included. 


Values for full AR, subset AR, full MA, and subset MA coeffi- 
cients, followed by the constant if there is one and the values 
of m and X for the power transform. 


Real scalar containing the value of noise variance. 


Integers containing the prediction origins to use. The val- 
ues must be at least maxp+maxq+l (maxp and maxq are 
the largest AR and MA lags in the expanded version of the 
model) and at most n, and tf must be less than or equal to 
Gils 


Integer containing the maximum number of steps ahead to 
forecast from each origin. 


Real scalar containing the confidence level for the probability 
limits to be placed on the forecasts (0<conf<1). 


Array of length (tl — tf + 1)h1 containing the forecasts. The 
first hl elements are from origin tf, the next hl are from 
origin tf+1, etc. 


Array containing the lower probability limits on the corre- 
sponding elements of xp. 
Array containing the upper probability limits on the corre- 
sponding elements of xp. 
Integer variable indicating whether SEASPRED finished with- 
out error (0) or an illegal power transform was requested (1). 
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Remarks: 


1. In the notation of Chapter 3, we have 


ords=<p,P,q,Q,M,d,D,S> 


Pe Workspace=5 (maxp+maxq+1)+3n+h1+3(t1-tf+1)h1, where the values of 
maxp and maxq are largest AR and MA lags when the model is expressed 
as an ARMA model. 


ee eee ee 


SIN Command 


MEET aeteneee = a Sa ee 


Purpose: 


Calculate sines of various kinds. 


Format: 
1. y=SIN(x) 
2. Y=SIN(X,n) 
3. Y=SIN(n,a,p) 


Remarks: 


1. In the first form, the output real scalar variable y is the sine of the real 
scalar input (expressed in radians) x. 


2. In the second form, the output array Y of length n contains the sines of 
the corresponding values in the input array X, which must be expressed in 
radians. 


3. In the third form, the elements of the output array Y of length n (n is 
an input integer) are given by Y(i)= asin(27(i — 1)/p), where the real 
scalars a and p are input and p#0. 


ee ee ee, AL 


SINGLEOFF Command 


a a ee ee 


Purpose: 


Cancel the effect of SINGLEON. 
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Format: 


SINGLEOFF 


Remark: 


1. See SINGLEON. 


SINGLEON Command 


Purpose: 


Signal TIMESLAB to henceforth wait for a carriage return before invoking 
any command in a macro; that is, execute macros in single step mode. 


Format: 


SINGLEON 


SORT Command 


Purpose: 


Form an array consisting of the elements of another array in ascending 
order. 


Format: 


Y=SORT(X,n) 


Input: 
X - Array of length n containing the data to be sorted. 


n - Integer containing the length of X. 


ny, - Array of length n containing the sorted data. 
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Ee eee 


SPEAKER Command 
ae 


Purpose: 
Make the PC’s speaker sound for a specified length of time in a specified 
frequency. 
Format: 
1. SPEAKER (freq, time) 
2. SPEAKER(freqs,times,n) 


Remarks: 


1. The first form turns on the speaker in frequency freq Hertz for time 
hundredths of a second. Both arguments are integers. 


2. The second form takes the two input arrays freqs and times of length n 
and successively turns on the speaker for times (i) hundredths of seconds 
in frequency freqs(i) Hertz. 


3. The middle C major scale is 523, 587, 659, 698, 784, 880, 988, 1047 Hertz. 


4. Frequencies must be between 21 and 65,535, and times must be between 
1 and 65,535. 


ee 


SPEAKEROFF Command 


ee eee 


Purpose: 


Cancel the effect of SPEAKERON. 


Format: 


SPEAKEROFF 


ee ee ee, 


SPEAKERON Command 


rp ee ee eee 


Purpose: 


Henceforth, any time a line is executed during a macro, the speaker beeps. 
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Format: 


SPEAKERON 


SUBMNS Command 


Purpose: 
Calculate and/or subtract and/or add seasonal means for a data set. 


Format: 


Y=SUBMNS (X,n,d,xbar[,iopt]) 


Input: 
X - Array of length n containing data to be operated upon. 
n - Integer containing the length of X (>0). 
d - Integer containing the period of the seasons (>0). 
iopt - Optional argument which if present is an integer indicating 
what operation is to be performed. The following options are 
available: 
0 Calculate seasonal means but don’t subtract them. 
1 Calculate and subtract seasonal means. 
2 Add inputted seasonal means xbar to the data (this 
allows for the inverse operation of iopt=1). 
xbar - (Only if iopt=2) Array of length d containing seasonal means 
to add to data. 
Output: 
¥ - (Unless iopt=0) Array of length n containing the result of 
subtracting or adding seasonal means. 
xbar - (Unless iopt=2) Array of length d (if d>1) or a real scalar 
variable (if d=1) containing seasonal means. 
Remarks: 


1. The argument iopt not being present is equivalent to iopt=1. 


2. Workspace=n+d. 
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SWEEP Command 


SS ee ee ee 


Purpose: 


To sweep a matrix. 


Format: 
1. B=SWEEP(A,n,k1 »K2,ier) 
2. B=SWEEP (A,n ,»ind,m,ier) 


Input: 
A - Array of length n*n containing the matrix to be swept. 
n - Integer containing the size of the matrix to be swept. 
k1,k2  - In the first form, A is swept on diagonals ki through k2. 
ind,m - In the second form, A is swept on the m diagonals whose 
indices are in the array ind. 
Output: 
B - Array of length n*n containing the result of the sweeping. 
ier - Integer variable containing an indicator of whether or not a 
zero diagonal was encountered during the sweeping (0 means 
no, 1 means yes). 
Remarks: 


1. In either form, the next diagonal to be swept on is the one having the 
largest absolute value of those remaining. 


2. If ier=1, then B is not formed. 


3. Workspace=n?+K, where K is the number of diagonals on which A is swept. 
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TEXTCOLOR Command 


a eee 


Purpose: 


Allow the user to set the foreground and background colors in the text 
mode. 
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Format: 
TEXTCOLOR (back, fore) 
Input: 
back - Integer containing the number of the color to be used for the 
background. The choices are those numbered 0 through 7 in 
the table of colors below. 
fore - Integer containing the number of the color to be used for the 
foreground. The choices are those numbered 0 through 15 in 
the table of colors below. 
Available Colors in the Text Mode 
Number Color Number Color Number Color 
1 Blue 6 Yellow-brown 11 Light blue-green 
2 Green 7 White 12 Light red 
3 Blue-green 8 Gray 13 Light purple 
4 Red 9 Light blue 14 Yellow 
5 Purple 10 Light green 15 Bright white 
Remarks: 


1. To get colors other than white on black, the user must have booted the PC 
with a file called CONFIG.SYS on the booting directory that contains the 
line DEVICE=ANSLSYS and ANSILSYS (which comes with DOS) must 
also be on the booting directory. 


2. TIMESLAB starts with white on blue. 


TIME Command 


Purpose: 

To obtain the number of seconds since midnight. This allows the user to 
time TIMESLAB operations. 
Format: 


TIME (t) 
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Remark: 


1. t is an output real scalar variable containing the time since midnight in 
seconds. TIME can only return the time to the nearest .05 second (approx- 
imately). 


TaBMNSTSFTiSiT MORRELL nite ee ee eee 


TOEPL Command 


ee eee 


Purpose: 


To form a symmetric Toeplitz matrix. 


Format: 


A=TOEPL(R,RO,n) 


Input: 
R - (If n>1) Array of length n-1 containing the second through 
nth elements of the first row of A. 
RO - Real scalar containing the value for the diagonal of A. 
n - Integer (1 < n < 100) containing the size of the resulting 
matrix. 
Output: 
A - Array of length n*n containing the desired matrix. 
Remarks: 


1. Even though the first argument is ignored if n=1, something must be 
included in the argument list. 


2. Workspace=n?2. 


aii: or ee ey 


TRANS Command 


Pea a ee ea ee a ee 


Purpose: 


Find the transpose of a matrix. 


Format: 


Y=TRANS (X,n,m) 
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Input: 
X - Array of length n*m containing the matrix to be transposed. 
n,m - Integers containing the dimensions of X. 
Output: 
NY, - Array of length n*m containing the transposed matrix. 
Remarks: 


1. The first n elements of X will be considered to be its first column, the next 
n will be the second column, and so on. 


2. Workspace=nn. 


TYPE2 Command 


TIMESLAB has a wide variety of assignment statements and arithmetic 
operations on integers, reals, and arrays. See Section B.4.3 for details on how 
these operations are performed. 


TYPE4 Command 


Purpose: 


To define an array by listing its elements or concatenating reals and/or 
arrays. Some examples are (see Section B.4.2 for more information): 


1. alpha=<1,2,3,4,5> 
eK =<1 2s 2k OL Oo> 


WHILE Command 


Purpose: 


To provide a WHILE...END construct in macros. 


Format: 


WHILE (var1.exp.var2) 
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Remark: 


THE TIMESLAB comMMANDS APP. C. 


1. The notation var1.exp.var2 represents an immediate or variable integer 
or real scalar followed by a decimal point, followed by one of the expres- 
sions 1t,le,eq,ge, or gt followed by another decimal point and another 
immediate or variable integer or scalar. All of this must occupy no more 
than 15 characters. If the expression is false, the next line of the macro to 
be executed is after the corresponding END. Otherwise it is the next line 
after the WHILE. 


Sa cee 


WINDOW Command 


ee 


Purpose: 


Calculate nonparametric spectral density estimates for univariate data. 


Format: 


f=WINDOW(rho,RO,M,Q, ioptw[,n,c]) 


Input: 


rho 


RO 
M 


ioptw 


Array of length M (if ioptw is between 1 and 5) or length n-1 
if ioptw is between 6 and 8 containing autocorrelations. 


Real scalar containing the sample variance (> 0). 
Integer (> 0) containing scale parameter. 


Number of frequencies between 0 and 1 at which to calculate 
spectra (the greatest prime factor of Q must be <23, and the 
product of its square-free prime factors must be <210). 


Integer containing the number of the window to be used in 


the estimation procedure (see the table below for the list of 
choices). 


(If either ioptw is between 6 and 8 or the factor for deter- 
mining confidence intervals is desired) Integer containing the 
length of the data set being analyzed. 
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Windows in the WINDOW Command 


ioptw Window ioptw Window 
1 Truncated periodogram 5 Bohman 
2 Bartlett 6 Daniell 
3 Tukey 7 Bartlett-Priestley 
4 Parzen 8 Parzen-Cogburn-Davis 
Output: 

f - Array of length [Q/2]+1 containing the spectral estimator at 
the frequencies (j-1)/Q, j=1,...,[Q/2]+1. 

c - Ifthis argument is present, it is a real scalar variable that can 
be used to find 95% confidence intervals for the true spectral 
density. The interval at the ith frequency would be from 
f£(i)/c to f (i) *c. 

Remark: 


1. Workspace=M+3Q. 


WN Command 


Purpose: 


Generate realizations from a white noise series. 


Format: 


X=WN (seed ,n, iopt) 


Input: 
seed - Real scalar containing the seed for the random number gen- 
erator. 
n - Integer containing the length of the desired realization. 
iopt - Integer containing the number of the distribution to use (see 


the table below for a list of choices). 
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Distributions in the WN Command 


iopt Distribution iopt Distribution 

1 N(0,1) 5 Standard Cauchy 

2 U(0,1) 6 Extreme value 

3 Unit exponential or Lognormal 

4 Logistic 8 Double exponential 
Output 

X - Array of length n containing the realization. 

Remarks: 


1. Leaving out iopt is the same as using iopt=1. 


2. Samples from each distribution are generated by transforming uniforms or 
normals. The uniforms are generated via the multiplicative congruential 
generator using multiplier 7° and modulus 231 —1 (see Lewis et al. (1969)) 
The normals are generated using the Marsaglia (1962) polar method. 


3. Workspace=n. 


APPENDIX D 


The Data Sets 


In this appendix we list the values for all of the data sets that are an- 
alyzed in this book. These listings are exactly how the data sets appear on 
distribution diskette number 2. Next to the heading for each data set is the 
name of the file containing the data. Note that the gas furnace data, New York 
City temperatures and births data, and the critical radio frequencies data are 
bivariate data sets. The univariate series in these three data sets can be read 
by the commands (taking the critical radio frequencies for example): 


READ (cradfq,x,n) 
READ (cradfq,y,n,1) 


The first ten data sets are Series I-X in Chapter 1. Note that several of 
the data sets have been coded by linear transformations. This has no effect on 
correlations or standardized spectral density plots. 

The index of this book contains a list of the page numbers where each 
data set is discussed in the book. 


Daily California Births: CALFEM.DAT 


This data set is an example of daily data. 


Female Births in California (1959) 
365 
35. 325 30, Sin 442 °29; 455 432 38) 27: 
38. 33 S5e 475 46: 37: 50% 43% 41% 52% 
S45'*O3; P*S9t" "S2iN S7t 432" 3921 3521 449 38% 
24° 2337 Sic? 4457 S82) 60x. SEN! SEIT Si8) SIS 
Sis) S65" 465 “SES S45) 620! 4795 467-1 4651 “SOF 
48> "ST. “SSS 5209428 453 “3988 3797 °S02 355 
28: “45: “S4F 362°°502" 4467 3959 325 (39% “462 
43°55. [S92 (SEF 272 302% 429° 1465" 41 436) “452 
46: -°433 “38 342 352) S6Ri°36s" 325° 50. 419 
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39. 
53. 
46. 
32. 
40. 
55. 
38. 
42. 
39. 
59. 
41. 
45. 
36. 
44. 
38. 
51. 
36. 
53. 
47. 
45. 
45. 
44. 
52. 
33. 
51. 
36. 
39. 
37. 


41. 
34. 
45. 
35. 
35. 
41. 
45. 
48. 
45. 
43. 
47. 
47. 
41. 
48. 
46. 
41. 
57. 
55. 
51. 
50. 
43. 
36. 
46. 
42. 
46. 
49. 
40. 
52. 


47. 
34. 
54. 
38. 
42. 
45. 
46. 
34. 
38. 
45. 
26. 
38. 
50. 
46. 
46. 
44. 
44. 
39. 
56. 
48. 
42. 
45. 
42. 
47. 
44. 
43. 
42. 
48. 


34. 
38. 
34. 
33. 
41. 
41. 
34. 
41. 
47. 
38. 
35. 
42. 
39. 
44. 
40. 
38. 
42. 
59. 
49. 
43. 
38. 
46. 
40. 
$1. 
52. 
43. 
42. 
55. 


36. 
35. 
37. 
39. 
42. 
37. 
35. 
35. 
33. 
37. 
44. 
35. 
41. 
37. 
39. 
68. 
53. 
55. 
54. 
40. 
47. 
48. 
40. 
44. 
45. 
34. 
53. 
50. 


DATA SETS 
33. 35. 38. 
32. 42. 34. 
35. 40. 42. 
47. 38. 52. 
38. 24. 34. 
435 3930033 
48. 51. 36. 
40. 34. 30. 
30. 42. 43. 
467 742) 572 
4127542) 36. 
36. 39. 45. 
46. 64. 45. 
39. 44. 45. 
44. 48. 50. 
40. 42. $51. 
42. 34. 40. 
73. 55. 44. 
56. 47. 44. 
59. 41. 42. 
38° 36. 427, 
49. 43. 42. 
45.935. 135. 
40. 57. 49. 
32. 46. 41. 
39. 35. 62. 
39. 40. 38. 


38. 
46. 
58. 
30. 
43. 
43. 
33. 
36. 
41. 
46. 
45. 
43. 
34. 
33. 
41. 
44. 
56. 
43. 
43. 
51. 
35. 
59. 
40. 
45. 
34. 
47. 
44, 


34. 
30. 
51. 
34. 
36. 
40. 
46. 
40. 
41. 
51. 
45. 
47. 
38. 
44. 
42. 
45. 
44. 
40. 
42. 
49. 
28. 
45. 
39. 
49. 
33. 
52. 
34, 


Beveridge Wheat Price Index: BEV.DAT 


This series has been analyzed extensivel 


Pp. 622, for example). 


Beveridge Wheat Price Index, 1500-1869 


3 


70 
17 
23 
20 
22 
18 


si 


1 


19 
18 
14 
16 
27 
37 
79 
76 
06 
97 
91 
99 
84 
71 
64 
79 
61 


1 


1 


1 


20 
17 
16 
19 
29 
41 
59 
13 
87 
77 
15 
96 
90 
94 
67 
74 
09 


25 


1 
1 


1 


15 
20 
5 
17 
36 
36 
47 
68 
77 
83 
99 
02 
20 
95 
71 
79 
08 


25 


1 
1 
1 


1 


13 
20 
8 
17 
29 
32 
48 
59 
77 
81 
99 
05 
24 
10 
72 
95 
10 


14 
18 
26 
19 
27 
47 
49 
74 
63 
82 
115 
114 
136 
154 
89 
70 
130 


14 
14 
26 
20 
30 
42 
45 
78 
70 
78 
101 
103 
120 
116 
114 
72 
166 


14 
16 
29 
24 
38 
37 
53 
69 
70 
75 
90 
98 
135 
99 
102 
63 
143 
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y in the past (see Anderson (1971), 


1 
1 


1 


14 
21 
20 
28 
50 
34 
55 
78 
63 
80 
95 
03 
00 
82 
85 
60 
03 


al | 
24 
18 
36 
24 
36 
55 
73 
61 
87 
108 
101 
70 
76 
88 
74 
89 


16 
15 
16 
20 
25 
43 
54 
88 
66 
72 
147 
110 
60 
64 
97 
75 
76 


19 
16 
22 
14 
30 
55 
56 
98 
78 
65 
112 
109 
72 
63 
94 
91 
93 
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82 71 69 75 134 183 113 108 121 139 109 90 
8 8&8 93 106 89 79 91 96 111 112 104 94 
98 88 94 81 77 84 92 96 102 95 98 125 
162 113 94 85 89 109 110 109 120 116 101 113 
109 105 94 102 141 135 118 115 111 127 124 113 
122 130 137 148 142 143 176 184 164 146 147 124 
119 135 125 116 132 133 144 145 146 138 139 154 
181 185 151 139 157 155 191 248 185 168 176 243 
289 251 232 207 276 250 216 205 206 208 226 302 
261 207 209 280 381 266 197 177 #170 152 156 141 
142 137 161 189 226 194 217 199 151 144 138 145 
156 184 216 204 186 197 183 175 183 230 278 179 
161 150 159 180 223 294 300 297 232 179 180 215 
258 236 202 174 179 210 268 267 208 224 


Normal White Noise Series: NORMWN.DAT 


This series was formed using the WN command. 


Normal White Noise Series 


106 

-0943 -.2142 - 8439 - 7062 -5208 - 2100 
1.4550 -4649 -.4410 =.9391 -.5790 -1.0580 
- .6708 -8435 -1.7748 - .8621 - .0782 -.2264 
- .5232 - .7443 -4134 - .3300 . 2164 ~-.1358 
- .6790 .8799 -1.4526 -9732 -0566 -1.4173 
-6477 - .2606 ~.4727 -.0500 - 8909 -6899 
-1.0235 1.2896 ~ .2868 -.7879 1.7034 +1273 
-.5479 - .6860 - 7058 - 2904 . 2679 1.8345 
2.5018 -.0287 -1.6743 1.3541 ~-.9202 1.1854 
-1.0682 - 0897 -.6558 -0990 -.9942 -5082 
- 4487 =, 1672 202477 - .6841 -3455 -.2513 
-1213 -.9045 -1.1407 = .2203 -.7520 -.7361 
-1348 -.7635 1.5830 -5722 -.7494 -1.5595 
1.0255 -.0823 -1.2096 1.7183 -1.0408 -1.0837 
-0935 1.8412 -6691 ~-1.0086 -1.7471 - .6900 
1.4420 -2133 -1.3858 =. 8704 8 =123716 - 9627 
-1.1039 ~.1556 1.1266 -4647 - .3326 -.0414 

2.1232 - .6909 . 2319 - .5646 


Annual Rainfall Data: RAINEAST.DAT 


This series is typical of many univariate meteorological data sets in that 
there is very little serial autocorrelation. 


Rainfall Eastern US, 1817-1922 


106 

22.54 -17.46 -26.46 11.54 6.54 -26.46 9.54 -1.46 -33.46 -14.46 
6.54 -9.46 6.54 -1.46 -1.46 -54 -6.46 -15.46 -5.46 4.54 
-7.46 -2.46 -6.46 -2.46 6.54 1.54 7.54 -9.46 -6.46 9.54 
10.54 -2.46 -.46 14.54 -13.46 1.54 -4.46 -5.46 -54 -13.46 


5.54 6.54 14.54 -7.46 -3.46 3.54 3.54 -14.46 16.54 -.46 
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2.54 12.54 
7.54 15.54 
-2.46 7.54 
3.54 5.54 
~.46 -12.46 
“11.46 -9.46 


DATA SETS 
6.54 13.54 1.54 -.46 
-2.46 4.54 3.54 6.54 
8.54 10.54 -54 .54 
-8.46 -7.46 -4.46 -54 
-5.46 -11.46 4.54 5.54 
9.54 3.54 -2.46 -54 


Artificial Series: ARTIF.DAT 


16.54 
9.54 
3.54 
3.54 

-3.46 


5.54 
6.54 
-6.46 
-9.46 
-9.46 


7.54 
3.54 
-12.46 
3.54 
6.54 
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15.54 
2.54 
5.46 
1.54 
-6.46 


See Section 1.5.2 for a description of how this data set was formed. 


Artificial Time 
100 
-5971 
-7160 
-7436 
-5911 
- 3866 
-7731 
. 2933 
-4382 
-5596 
.0752 
-2265 
.8087 
-9917 
-5537 
-0249 
-4343 
-4240 


Series 
12.2084 13 
11.4201 13 
10.2038 9 
13.5058 14 
10.0452 9 
15.1213 13 
11.5182 13 
12.9454 11 
20.1978 18 
12.6914 12 
18.5547 19 
16.1202 16 
14.7829 15 
20.0116 19 
18.1552 18 
22.0911 19 
17 .3307 20 


International Airline Data: 


This series is one of the most often anal 


example of the Box- 


Airline Passenge 


144 
1949 112 118 
1950 115 126 
1951 145 150 
1952 171 180 
1953. 196 196 
1954 204 188 
1955 242 233 
1956 284 277 
1957 315 301 
1958 340 318 
1959 360 342 
1960 417 391 


rs, 1949-60 


(5X,12F5.0) 


132 
141 
178 
193 
236 
235 
267 
317 
356 
362 
406 
419 


129 
135 
163 
181 
235 
227, 
269 
313 
348 
348 
396 
461 


121 
125 
172 
183 
229 
234 
270 
318 
355 
363 
420 
472 


-0362 10.1389 8.8044 
-0817 14.1539 14.0661 
-7150 9.7878 10.0856 
-8224 14.0889 13.0398 
-1542 12.0071 13.9843 
-4067 10.9186 8.1324 
-5241 14.6620 17.4796 
-2046 10.8988 12.0149 
-5038 19.2840 16.5496 
-6856 14.5315 15.5767 
-3905 16.4114 15.6677 
-8718 18.4903 18.6455 
-3884 15.4088 15.1301 
-5788 21.2045 20.3659 
-8584 18.8034 18.7778 
-1892 16.8242 14.4648 
-0159 21.5673 
AIR.DAT 


135 
149 
178 
218 
243 
264 
315 
374 
422 
435 
472 
535 


148 
170 
199 
230 
264 
302 
364 
413 
465 
491 
548 
622 


148 
170 
199 
242 
272 
293 
347 
405 
467 
505 
559 
606 


- 5840 
- 5436 
+2560 
-1967 
-1276 
-5325 
-6391 
-5140 
- 9002 
-9975 
-4860 
-8001 
. 2064 
-6175 
-9501 
- 6066 


yzed data sets. It is the classic 
Jenkins multiplicative seasonal model. 


136 
158 
184 
209 
237 
259 
312 
355 
404 
404 
463 
508 


119 
133 
162 
191 
211 
229 
274 
306 
347 
359 
407 
461 


104 
114 
146 
172 
180 
203 
237 
271 
305 
310 
362 
390 


118 
140 
166 
194 
201 
229 
278 
306 
336 
337 
405 
432 
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Random Walk Series: RW.DAT 


DATA SETS 
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This series was formed by the cumulative sums of a normal white noise 


series. 


Random 
200 


ONADOALNA OO 


ll 
ArPrRrEFR OD We 


Walk 


- 7859 
- 3520 
-3214 
-4549 
-1631 
-4150 
- 6346 
-4833 
- 7938 
-2219 
-5535 
-0351 
- 9900 
. 7099 
. 2093 
. 8876 
- 6076 
- 3943 
- 6300 
-5092 
-9072 
- 7322 
-8078 
-6311 
-0999 
-9447 
- 7496 
-5817 
- 5665 


6.4711 5.6384 
7.5554 
6 .9438 
5.6714 


1 


on 
CaN NAAAWH © 


- 
rs 


ee 
woOne DH WwW 


i 
a 


3. 
5 


7457 
- 9904 
- 9486 


0.8089 


nN 
nN 
Ww 
Ww 
N 


. 3240 
-1621 
-9819 
- 6684 
-7154 
-9196 
-5348 
- 9628 
- 7365 
-4062 
- 2653 
-1622 14.4308 
-5721 12.6238 
-4243 12.8397 
-9441 14.6993 
-5200 15.6034 


9.4746 
6.8656 
5.5356 
4.3934 
6.8264 
6.9439 
12.1630 
9.8986 
13.4497 


ODAAMDAANAMNM HA O 


ee 
Ny WwW 


-9523 
-9112 
- 5003 
- 4536 
-8295 
- 7677 
- 1584 
- 5953 
- 7333 
-4524 
-1451 
-5885 
-4716 


16.0370 


4.5445 
8.8956 
6.6391 
4.6291 
4.1329 
6.5395 
7.9076 
11.2429 
10.4068 
13.5710 
9.1888 
5.7016 
8.4917 
4.0820 
5.7880 
7.6116 
7.4693 
6.1226 
6.4188 
7.4116 
9.7785 
13.2428 
13.3632 
15.5422 
12.3841 
12.2099 
11.3166 
15.1034 


5.2512 
7.8375 


6.1159 
5.5859 


16.5936 


Levels of Lake Erie: ERIEL.DAT 


3.5110 
5.7517 
8.1756 
12.4282 
10.8555 
12.6664 
10.7718 
7.4199 
6.7446 
5.0447 
7.9112 
8.3178 
5.6946 
6.0727 
7.1246 
6.3278 
10.0496 
15.0027 
13.6777 
14.7332 
13.1813 
13.1526 
12.2531 
15.4357 


5.4320 
7.1667 
5.3716 
4.1332 
3.6592 
7.4551 


10.0101 
11.3600 
10.6883 
11.5258 
11.3440 


7.0175 
6.0545 
3.9409 
7.2202 
8.8387 
4.8324 
6.9636 
7.4150 
7.4545 
9.8638 


15.2795 
13.4282 
14.2666 
12.8510 
14.0196 
14.1968 
16.3698 


owann a» 


a a od 
or N KB NY 


NN ANMWAONW AAD 


le ed 
aA nrkn WO PO 


. 8870 
-0101 
- 7850 
-1065 
- 4896 
-1824 
-5119 
-4497 
-9060 
-3055 
-5945 
- 8965 
- 4966 
- 7853 
+4522 
- 0486 
-6357 
-6535 
- 7606 
-9191 
-9912 
-6536 
-5994 
- 9666 
-5237 
-3557 
-2011 
-1636 


This series has both short and long term rises and falls. 


Lake Erie Levels 1921-1970 


600 


14. 
14. 
-446 
14. 
15. 
p 
-157 
13. 
14. 


15 


11 


763 
478 


858 
161 
827 


852 
156 


14. 
14) 
14. 
14. 
is. 
13. 
10. 
13. 
13. 


649 
364 
649 
288 
427 
567 
892 
833 
586 


15 


13. 
13. 
13. 
15. 
13. 


11 


13. 
13. 


-085 
928 
776 
947 
693 
548 
-120 
169 
150 


16. 
13. 
13. 
13. 
15. 
13. 
2% 
12 
14. 


376 
283 
188 
416 
142 
302 
600 
941 
137 


16 
13 


12 
14 


13 


14 


-926 
-700 
13. 


283 


-903 
-763 
13. 


188 


-283 
13. 
-231 


188 


16 
15 


2s 
-454 
14. 
13. 
13. 
14. 
14. 


13 


-774 
-465 


657 


288 
112 
416 
383 
364 


16 


12 
13 
13 
12 
13 
14 
13 


-490 
16. 


243 


-979 
-491 
-074 
-827 
-340 
-763 
-833 


15. 
16. 
13. 
13. 
de 
¥2. 
13. 
15. 
14. 


769 
490 
909 
567 
543 
201 
529 
218 
478 


15 


13 


11 
13 
15 
15 


-180 
16. 
14. 
.397 
5 
-917 
-776 
-161 
.009 


243 
535 


239 


14. 
15. 
14. 
14. 
-O11 
-803 
14. 
14. 
-617 


12 
11 


15 


383 
787 
877 
440 


307 
858 
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16.148 15.977 15.142 14.592 14.364 14.497 14.554 14.991 15.863 17.932 
19.184 19.184 18.956 18.254 17.514 16.660 16.338 16.319 17.457 17.173 
17.856 18.596 18.558 18.159 17.685 16.812 16.072 15.332 14.478 14.213 
13.738 13.169 12.581 13.245 13.852 14.175 14.288 13.985 13.435 12.884 
12.429 12.410 13.397 13.909 13.833 14.099 14.687 14.611 14.383 13.909 
13.359 12.296 12.106 11.803 12.353 12.220 12.827 14.250 15.085 14.953 
14.440 13.890 13.036 12.201 11.404 11.309 10.987 10.361 10.304 11.347 
11.784 11.841 11.841 11.651 11.404 10.873 10.209 10.076 10.247 10.133 
10.740 11.556 12.201 12.505 12.732 12.201 12.068 11.290 11.139 11.101 
10.342 10.000 11.347 12.770 13.321 13.340 13.188 12.676 12.315 12.049 
11.594 11.252 12.467 13.491 13.491 14.156 15.256 15.598 16.034 15.598 
14.516 13.435 12.638 12.106 12.144 12.979 11.917 15.066 15.199 15.427 
15.427 15.408 14.706 14.137 13.302 12.979 13.036 12.903 13.719 14.801 
15.541 15.617 15.503 15.218 14.592 13.852 13.416 13.055 12.315 12.239 
12.562 13.890 14.687 15.313 15.408 15.028 14.782 14.213 13.435 13.662 
14.288 13.491 13.150 13.586 13.871 14.175 14.099 13.719 13.093 12.562 
12.030 12.144 11.860 12.410 12.770 14.630 15.218 15.920 15.882 15.750 
15.275 14.801 14.554 14.345 15.104 14.554 14.953 15.958 17.590 18.805 
18.805 18.330 17.476 16.812 16.224 15.541 14.744 14.478 14.725 16.319 
17.324 17.609 17.211 16.546 15.977 15.427 14.972 14.535 14.213 13.719 
15.009 16.319 17.078 17.913 18.159 17.704 17.059 17.268 16.546 16.072 
15.996 15.294 15.901 16.300 16.546 17.495 17.666 16.964 16.148 15.427 
14.839 14.269 14.118 14.156 14.080 16.414 18.216 19.298 18.824 18.368 
17.875 16.869 16.129 15.712 15.617 15.161 16.148 17.609 18.406 18.463 
18.254 17.609 16.812 15.712 15.066 14.763 14.877 15.370 15.731 15.996 
16.224 16.186 16.015 15.446 14.573 14.080 13.226 12.979 14.459 15.655 
15.636 17.154 17.381 17.116 16.736 16.167 15.920 15.408 15.066 15.769 
15.787 15.863 17.154 18.178 18.653 18.615 18.406 17.837 17.078 16.471 
16.224 16.395 17.400 18.672 19.089 19.829 20.000 19.943 19.526 18.975 
18.463 17.268 16.414 16.414 16.755 16.736 17.173 17.647 18.216 18.767 
18.539 18.273 17.419 16.679 15.996 15.465 15.408 15.123 16.072 17.685 
18.235 18.064 17.818 17.438 16.812 17.116 17.211 17.097 17.495 17.097 
18.121 18.748 18.767 18.539 18.102 17.818 17.002 16.357 15.560 15.313 
14.782 13.719 14.839 15.825 17.324 17.723 17.685 17.514 16.907 15.863 
14.934 14.706 14.383 14.345 14.763 15.958 16.698 16.793 17.230 16.509 
15.787 14.991 14.080 14.326 14.497 13.510 13.662 14.042 14.269 14.478 
14.972 14.972 14.573 13.776 13.017 12.600 12.296 13.131 13.966 15.028 
15.844 15.769 15.237 14.801 14.137 13.890 13.416 13.871 14.478 14.725 
14.763 15.806 16.565 17.097 17.306 17.211 16.717 15.787 14.801 14.231 
13.966 14.004 15.313 16.357 17.742 17.609 17.249 17.078 16.509 15.465 
14.706 14.213 13.662 13.928 14.516 15.180 15.351 15.579 15.446 15.199 
14.725 14.383 14.231 13.776 13.150 12.713 13.283 14.478 14.858 14.782 
14.307 14.023 13.529 12.922 12.410 11.936 11.784 11.992 12.619 13.662 
14.231 14.099 13.833 13.416 12.922 11.974 11.461 11.423 11.860 12.163 
13.226 13.966 14.535 14.516 14.231 13.966 13.738 13.226 12.998 13.131 
13.700 13.814 14.383 15.047 15.693 15.844 15.712 15.313 14.763 13.548 
13.586 14.516 14.383 14.497 14.744 15.806 16.527 16.546 16.717 16.433 
15.769 15.275 15.123 15.541 15.825 16.376 16.357 16.907 17.021 17.419 
17.533 17.268 16.603 15.825 15.446 15.636 15.693 16.755 16.509 17.723 
18.691 19.127 19.564 19.203 18.216 17.211 16.660 16.831 15.769 15.731 
15.996 17.021 17.552 17.837 17.856 17.571 17.078 16.660 16.433 16.584 
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DATA SETS 


Sum of Four Cosine Curves: COS4.DAT 


This series was formed by the macro in Example 1.4. 


Sum of Four Cosine Curves 


144 


16.0000 
14.0981 
17.0981 
10.0000 
11.9019 

8.9019 
16.0000 
14.0981 
17.0981 
10.0000 
11.9020 

8.9020 
16.0000 
14.0981 
17.0981 
10.0000 
11.9020 

8.9020 
16.0000 
14.0981 
17.0981 
10.0001 
11.9020 

8.9019 


. 5679 
- 7592 
-7175 
-9077 
-9011 
-0573 
. 5679 
-7592 
-7175 
- 9077 
+9011 
-0573 
. 5678 
- 7592 
-7175 
. 9078 
-9011 
-0573 
- 5678 
- 7592 
-7175 
- 9078 
+9009 
-0575 


-3.5343 


-9.5681 


4.3377 
14.6619 
-8.6281 
13.8585 


-3.5343 


-9.5681 
-4.3377 
14.6619 
-8.6281 
13.8585 
-3.5342 
-9.5681 
4.3377 
14.6619 
-8.6281 
13.8585 
-3.5342 
-9.5681 
-4.3377 
14.6619 
8.6281 
13.8584 


i 


is 


-0000 
-6339 


Gas Furnace Data: GASFURN.DAT 


- 3906 


= 2.7192 
-2.2175 
-8.1944 
-5 .0847 


- 5864 
- 3906 
+7192 
-2175 


-8.1944 


-0847 
-5864 
. 3906 
-7192 
+2175 


-8.1944 


-0847 
- 5864 
- 3905 
-7192 
-2174 


-8.1944 


-0847 
- 5864 


ae 
-1210 
.5260 
- 5390 
-5392 
-1342 
-1212 
-1210 
-5260 
- 5390 
-5392 
-1342 
-1212 
-1210 
-5260 
-5389 
-5392 
-1342 
Sr > 
-1210 
+5259 
-5390 
-5392 
+1342 


1213 


This bivariate time series has been analyzed by both Box 


(1970) and Jenkins and Watts (1968). 


Input Gas Rate 


296 


Stit 

-13 
~-.47 

-99 
1.61 

- 96 
1.21 

-O1 
1.22 
-1.55 
-.96 
a2 
1.05 
eae 


.34 


+44 


S8ege 


-91 
-1.46 
-1.89 

-94 


+25 


=i. 


Si. 


Ais 


585 


and Jenkins 
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=2.28° =2.69 §-2°72.0) -o75i) | =179N=1 a5 -1.08 ~-.91 
~ .88 -.88 =(50 -.54 =42 ~.27 00 40 
84 1.28 1.61 1.75 1.68 1.49 99 65 
58 58 63 75 -90 99 97 79 
40 -.16 -.55 -.60 =.42 -.19 -.05 06 
16 30 52 -57 -56 57 59 67 
93 1.34 1.46 1.35 77 22 -.24 Sent 
=1 lO — 1 21d 7, - .68 -03 56 64 48 
i1 -.31 =. (Ose 105) 9=12998 T=1hi3 =.8% -.34 
06 08 00 -00 -21 56 78 86 
92 86 42 -.34 ~.96 -1.81 -2.38 -2.50 
“2.47 =2.33 -2.05 <-1.74 -1.26 =.57 -.14 =!102 
-.05 -.14 -.28 -.53 ~.87 -1.24 -1.44 -1.42 
<1.17 —7OL -.63 -.58 - .63 Sie! -.85  -1.04 
335° 6) 1,63 = 1.62 tet 5 = 49) =~ 16 = OL =.09 
~-62 -1.09 -1.52 -1.86 -2.03 -2.02 =1.96, =1'-95 
-1.79 -1.30 -1.03 Sey! -.80 =~ 8% 105) 1.12 
-.88 -.40 19 -66 71 61 50 60 
94 1.22 1.25 -82 -10 03 38 92 
1.03 87 53 09 -.46 -.75 -.95 -1.03 
-.93 -.64 -.42 -.28 -.16 -.03 10 25 
28 00 =49 =276 -.82 -.74 -.53 -.20 
-03 -20 -25 -19 -13 -02 -.18 -.26 
Output C02 
296 
53.8 53.6 53.5 53.5 53.4 53.1 52.7 52.4 52.2 52.0 52.0 52.4 
53.0 54.0 54.9 56.0 56.8 56.8 56.4 55.7 55.0 54.3 53.2 52.3 
51.6 51.2 50.8 50.5 50.0 49.2 48.4 47.9 47.6 47.5 47.5 47.6 
48.1 49.0 50.0 51.14 51.8 51.9 51.7 51.2 50.0 48.3 47.0 45.8 
45.6 46.0 46.9 47.8 48.2 48.3 47.9 47.2 47.2 48.1 49.4 50.6 
51.5 51.6 51.2 50.5 50.1 49.8 49.6 49.4 49.3 49.2 49.3 49.7 
50.3 51.3 52.8 54.4 56.0 56.9 57.5 57.3 56.6 56.0 55.4 55.4 
56.4 57.2 58.0 58.4 58.4 58.1 57.7 57.0 56.0 54.7 53.2 52.1 
51.6 51.0 50.5 50.4 51.0 51.8 52.4 53.0 53.4 53.6 53.7 53.8 
53.8 53.8 53.3 53.0 52.9 53.4 54.6 56.4 58.0 59.4 60.2 60.0 
59.4 58.4 57.6 56.9 56.4 56.0 55.7 55.3 55.0 54.4 53.7 52.8 
51.6 50.6 49.4 48.8 48.5 48.7 49.2 49.8 50.4 50.7 50.9 50.7 
50.5 50.4 50.2 50.4 51.2 52.3 53.2 53.9 54.1 54.0 53.6 53.2 
53.0 52.8 52.3 51.9 51.6 51.6 51 4 51.2 50.7 50.0 49.4 49.3 
49.7 50.6 51.8 53.0 54.0 55.3 55.9 55.9 54.6 53.5 52.4 52.1 
52.3 53.0 53.8 54.6 55.4 55.9 55.9 55.2 54.4 53.7 53.6 53.6 
53.2 52.5 52.0 51.4 51.0 50.9 52.4 53.5 55.6 58.0 59.5 60.0 
60.4 60.5 60.2 59.7 59.0 57.6 56.4 55.2 54.5 54.1 54.1 54.4 
55.5 56.2 57.0 57.3 57.4 57.0 56.4 55.9 55.5 55.3 55.2 55.4 
56.0 56.5 57.1 57.3 56.8 55.6 55.0 54.1 54.3 55.3 56.4 57.2 
57.8 58.3 58.6 58.8 58.8 58.6 58.0 57.4 57.0 56.4 56.3 56.4 
56.4 56.0 55.2 54.0 53.0 52.0 51.6 51.6 51.1 50.4 50.0 50.0 
52.0 54.0 55.1 54.5 52.8 51.4 50.8 51.2 52.0 52.8 53.8 54.5 
54.9 54.9 54.8 54.4 53.7 53.3 52.8 52.6 52.6 53.0 54.3 56.0 
57.0 58.0 58.6 58.5 58.3 57.8 57.3 57.0 


APP. D. DATA SETS 587 


Sales Data: SALES.DAT 


This series is discussed extensively in Sections 3.6 and 3.7. 


Sales of Company X, 1/1965 - 5/1971 

ae 
154 96 73 49 36 59 95 169 210 278 298 245 
200 118 90 79 78 91 167 169 289 347 375 203 
223 104 107 85 75 99 135 211 335 460 488 326 
346 261 224 141 148 145 223 272 445 560 612 467 
518 404 300 210 196 186 247 343 464 680 711 610 
613 392 273 322 189 257 324 404 677 858 895 664 
628 308 324 248 272 


Wolfer Sunspot Data: WOLFER.DAT 


This is another classic data set in the literature of time series analysis (see 
Morris (1977) for example). 


Wolfer Sunspot Numbers, 1749-1963 


215 
SO29" PSSi4 TAT TC (AT -8 FoS0% 81222 9.6 10.2 32.4 47.6 
54.0 62.9 85.9 61.2 45.1 36.4 20.9 11.4 37.8 69.8 
106.1 100.8 81.6 66.5 34.8 30.6 7.0 19.8 92.5 154.4 
125.9 684.8 68.1 38.5 22.8 10.2 24.1 82.9 132.0 130.9 
118.1 89.9 66.6 60.0 46.9 41.0 21.3 16.0 6.4 4.1 
6.8 14.5 34.0 45.0 43.1 47.6 42.2 28.1 10.1 8.1 
2.5 0.0 1.4 5.0 12.2 13.9 35.4 45.8 41.1 30.4 
23.9 915 57 6.6 4.0 1.8 8.6 16.6 36.3 49.7 62.5 
6720 4471.0) 554758) 4427-5 856 gel3225)66 59 912135 9138235010352 
8528 463.2 6:36.8. 662452. 91057) 611520609 40-48 16125 9983521124.3 
95.9 66.5 64.5 54.2 39.0 20.6 6.7 4.3 22:8 54.8 
93.8 95.7 77.2 59.1 44.0 47.0 30.5 16.3 123) 37.3 
73.9 13951 911222 101.70 “6653: 4477—" 17.1 i173: 91223 3.4 
G20 DES25See> 4. Son 59.7 = 63505 G326 52.2 §2524 4% 13.1 6.8 
6.3 (ea, seSOGOr™ (3.0) O4.9) 978.0) 64.0 (41288 | 26.2. 26.7; 
12.2 9.5 237 §.0 24.4 42.0 63.6 53.8 62.0 48.5 
43.9 18.6 5.7 3.6 1.4 9:6. 47.4 57.1. 103.9 80.6 
63.6) (37-6 5 26-1 14.2 5.8 16,7. 44-3. 163.9) «69'.0.5 77.8 
Gh.OR S05. eee loree th. ft 5.7 8.7 36.1 80.4 114.4 109.6 
88:8) 467-5) 9540.05050165) a165S 3.65 33-2) 49236 154:5681136.3 
134.7 83:59 » 69.45 31.5 13:9 4.4 38.0 141.7 190.2 184.8 
1590 4125389 553.999 37.58) 2779 


Lynx Data: LYNX.DAT 


This classic data set is analyzed in several sections of Chapter 3. 
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LYNX DATA 
114 
269. 321. 585. 871. 1475. 2821. 
3928. 5943. 4950. 2577. 523. 98. 
184. 279. 409. 2285. 2685. 3409. 
1824. 409. 151. 45. 68. 213. 
546. 1033. 2129. 2536. 957. 361. 
377. 225. 360. 731. 1638. 2725. 
2871. 2119. 684. 299. 236. 245. 
552. 1623. 3311. 6721. 4245. 687. 
255. 473. 358. 784. 1594. 1676. 
2251. 1426. 756. 299. 201. 229. 
469. 736. 2042. 2811. 4431. 2511. 
389. 73. 39. 49. 59. 188. 
377. 1292. 4031. 3495. 587. 105. 
153. 387. 758. 1307. 3465. 6991. 
6313. 3794. 1836. 345. 382. 808. 
1388. 2713. 3800. 3091. 2985. 3790. 
674. 81. 80. 108. 229. 399. 
1132. 2432. 3574. 2935. 1537. 529. 
485. 662. 1000. 1590. 2657. 3396. 


Magnitude of a Star: STAR.DAT 


This data set consists of the brightness of a variable star on 600 successive 


midnights (see Kedem (1986)). See Figures 3.8 and 3.13 for more information 
about this data set. 


MAGHITUDE OF A STAR 
600 (20F3.0) 
25 28 31 32 33 33 32 31 28 25 22 18 14 10 Ue Cy ee ty HG) 
2 4 8 11 15 19 23 26 29 32 33 34 33 32 30 27 24 20 17 13 
10 7 45°43 73°53 V4" 5 7 1013) 16 19109 24 26 27 28 29 28 
27 25 24 21 19 17 15 13 12 11 11 10 10 11 12 12 13 14 15 16 


APP. D. 


DATA 


SETS 


23 26 28 30 31 31 31 29 27 24 2219161311 9 8 7 7 7 
8 9 11 12 14 16 18 20 21 22 23 23 23 23 23 22 21 20 19 18 
18 17 17 16 16 16 16 15 15 15 14 14 13 13 13 13 13 13 14 14 
15 16 18 19 21 22 24 24 25 26 26 25 24 23 21 1916 1412 9 
7 5 5&5 4 & 6 8 10 13 16 20 23 26 29 31 32 32 32 31 29 
26°23 20016712 73.2693. 3) 0970 (Lies '6510 13 17221 25528 
31 33 34 34 33 31 29 26 22 18 15 11 8 5 3 222 4 & 


Mauna Loa Carbon Dioxide Data: MLCO2.DAT 


589 


This data consists of 216 monthly measurements of carbon dioxide above 
Mauna Loa, Hawaii. 


Mauna Loa Carbon Dioxide 


216 


14. 
13. 
14. 
18. 
20. 
19. 
18. 
17. 
ah ee 


21 


23. 
a2 
247 3 
20. 
22% 
26. 
29. 
28. 
29. 
28. 
ats 
32. 


88 
64 
12 
15 
47 
74 
80 
o1 
13 
-53 
57 
50 
15 
73 


tre 
14. 
13. 
16. 


19 


20. 
19). 
18. 


62 
71 
58 
00 
i 
63 
08 
36 
-02 
-13 
+29 
-00 
-73 
+25 
-67 
-92 
-04 
ane 
-02 
-07 
-85 
-40 


16. 


15 


15. 
14. 
18. 


21 


20. 
19. 
17. 
177. 
22. 
24. 
23. 


23 


oe He 
23. 
vy Oe 
30. 
30. 
29. 


28 


28. 


33 
-62 
14 
23 
78 
-21 
15 
37 
84 
99 
36 
46 
50 
03 
61 
49 
87 
58 
95 
84 
-79 
89 


-59 
-59 
AU 
-07 
-84 
-83 
-49 
-93 
-78 
-70 
ond 
-46 
+52 
-53 
-07 
-50 
-00 
-48 
+95 
ory 
-13 
-55 


17.93 
16.94 
16.62 
15.04 
15.16 
19.55 
22.25 
20.40 
19.55 
19.15 
17.89 
22.19 
25.11 
25.62 
24.91 
24.34 
24.06 
28.56 
32.85 
31.94 
30.64 
29.57 


Buffalo Snowfall Data: BUFFSNOW.DAT 


14.02 
16.67 
20.02 
18.63 
18.06 
16.84 
16.25 
20.49 
22.35 
22.13 
21.80 
20.24 
19.89 
25 .63 
27.97 
27.56 
27 .30 
26.97 
28.12 
31.79 
33.76 


12.83 
14.96 
19.74 
19.43 
18.59 
17.70 
16.17 
18.48 
22.19 
23.30 
22.03 
21.59 
19.80 
23.28 
2th: 
28.23 
28.20 
28.09 
27 .67 
29.61 
33.74 


This data set has been analyzed frequently, not so much from a time series 
anlysis point of view, but from a probability density estimation point of view 
(see Problem C1.24, Parzen (1979), and Silverman (1986), p. 44). 


Yearly Snowfall in Buffalo 1910-1972 


63 
1 


26.4 
69.3 
79.0 
77.8 
88.7 


82.4 
53.5 
74.4 
79.3 
71.4 


78.1 
39.8 
49.6 
89.6 
83.0 


115.6 102.4 101.4 


51.1 
63.6 
54.7 
85.5 
55.9 
89.8 


90.9 
46.7 
71.8 
58.0 
89.9 
71.5 


76.2 
12.9 
49.1 
120.7 
84.8 
70.9 


104.5 

79.6 
103.9 
110.5 


87.4 
83.6 
51.6 
65.4 


110.5 
80.7 
82.4 
39.9 


25.0 
60.3 
83.6 
40.1 


105.2 113.7 124.7 114.5 


98.3 


55.5 


66.1 


78.4 
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120.5 97.0 110.0 
Critical Radio Frequencies: CRADFQ.DAT 


The first univariate series in this bivariate data set consists of the median 
noon hour value of what are called critical radio frequencies each month in 
Washington, D.C., for the period May 1934 to April 1954. These frequencies 
teflect the highest radio frequency that can be used for broadcasting. See 
Siddiqui (1962) for further discussion of these data. The second univariate 
series consists of the total number of sunspots for the corresponding months. 
See Sections 3.8 and 3.9 for a discussion of these series. 


Critical Radio Frequencies (f0F2) 


240 
5.40 5.40 5.10 4.80 5.50 6.10 6.40 6.80 
6.40 6.00 6.10 6.10 5.30 5.50 5.70 5.00 
5.80 9.00 10.40 9.90 10.70 9.70 8.80 8.10 
8.40 6.40 6.70 7.40 9.10 11.90 12.50 11.80 
11.10 13.10 12.30 9.70 7.70 7.60 8.10 8.90 
10.40 13.00 13.20 12.30 11.50 12.80 12.20 10.00 
8.10 7.30 7.40 7.90 9.60 12.10 13.10 12.40 
11.10 11.70 10.30 9.30 8.00 7.40 6.90 7.60 
8.90 12.20 11.40 10.30 9.50 9.60 9.80 8.40 
40)) 6.708 16-40" 72400) 19°20 10.70 10.60 9.30 
9.00 8.90 8.10 7.20 5.70 5.80 5.80 6.20 
6.90 9.10 9.00 8.70 8.00 7.80 8.70 7.40 
6.10 5.30 4.90 5.10 6.00 7.40 7.50 7.00 
6.40 6.60 6.20 5.60 5.10 5.20 5.20 4.80 
5.20 6.00 6.60 7.10 6.40 6.00 5.90 5.20 
4.90 4.70 4.70 5.00 5.60 7.00 6.60 7.10 
6.50 6.60 6.20 6.30 5.70 5.80 5.50 5.70 
6.40 9.30 9.30 8.60 7.70 9.40 9.70 9.20 
6.20 6.30 6.10 7.20 8.60 11.50 12.20 12.00 
12.00 13.00 12.50 11.20 10.00 6.90 aly resres) 
10.60 13.00 13.50 12.60 12.30 11.30 11.20 10.80 
8.20 7.60 7.10 6.80 9.20 11.50 12.40 11.70 
12.10 12.50 12.50 10.60 8.30 7.00 7.10 7.80 
10.10 12.60 13.10 12.00 11.30 11.00 10.50 8.80 
7.20 6.80 6.00 6.10 7.00 9.20 9.00 8.60 
8.40 8.70 7.80 6.30 6.10 6.00 5.60 5.80 
7.00 8.60 9.20 8.80 8.00 7.50 6.50 5.50 
4.80 5.20 5.00 5.30 6.00 7.20 7.60 7.30 
6.40 6.20 5.20 5.20 4.80 4.80 4.30 4.70 
6.00 6.20 6.50 6.30 5.90 5.70 5.50 5.00 


Monthly Sunspot Wumbers 5/34-4/54 

240 
15.9850 15.3350 15.4650 15.4150 15.2000 15.2850 
15.4350 15.7700 15.9300 16.0250 16.1550 15.6100 
16.3650 17.2850 16.6950 16.5050 17.1050 17.6600 
18.2100 18.0750 18.1400 18.7150 18.8550 18.7450 
17.7300 18.5000 17.6150 19.3500 18.8000 19.4500 
20.7700 21.1700 21.6250 21.4250 19.1950 20.4650 
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LH Data: LH.DAT 


20.8350 
18.7200 
21.3700 
21.1100 
20.9150 
18.4050 
17.7200 
17 .9200 
16.4750 
16.9200 
16.2500 
16.5350 
15.7050 
15.5100 
15.1250 
15.5400 
16.5300 
17 .3000 
19.2450 
21.1900 
25.0650 
21.4000 
23.7000 
19.7900 
20.3100 
22.1750 
20.3100 
17 .7400 
20.4250 
17 .6200 
16.1700 
16.1050 
15.6250 
15.0800 


21 


15 


21 


21 
21 


17 


-5150 
19. 
i9. 
19. 
20. 
Lv 
19. 
18. 
17. 
16. 
15. 
16. 


4400 
8750 
6350 
0500 
1050 
1950 
4150 
9900 
6850 
5700 
1250 


-3800 
15. 
15. 
16. 
16. 
16. 
18. 
-0850 
23. 
20. 
23. 
-9000 
-0850 
20. 
19. 
-7050 
20. 
17. 
16. 
16. 
16. 
15. 


9400 
2500 
4200 
8100 
3700 
6750 


1950 
8250 
3900 


8800 
1800 


0300 
2900 
8200 
7150 
0900 
1250 


DATA SETS 
22.2550 21.8850 
19.9200 20.9600 
23.2650 20.7850 
19.0150 18.8700 
19.8800 20.2900 
17.5250 17.9700 
18.3750 20.2750 
17.2800 17.2250 
18.3450 18.0000 
16.7800 17.6400 
15.8850 16.0100 
15.6200 16.4450 
15.6600 15.9700 
15.1850 15.0250 
15.2500 15.8350 
15.9250 15.6350 
17.1300 16.2950 
17.3800 19.3100 
20.8100 20.3600 
20.7850 21.6700 
22.8950 24.4400 
20.4250 19.3050 
22.1100 22.8950 
20.9550 24.1150 
21.2900 21.1900 
20.0800 19.7400 
19.5500 19.2600 
17.9950 17.9950 
18.0750 18.0500 
17.0350 16.1350 
16.9650 17.7450 
16.3250 15.1950 
15.4300 16.1750 
15.0100 15.0250 


-0350 
-3250 
-4800 
+2300 
-6300 
-1650 
-3250 
-3200 
-2950 
-7100 
-8600 
-3700 
-5000 
-5500 
-7150 
-0750 
-7450 
-8300 
-7200 
-4900 
-4700 
-7400 
-1650 
-8750 
-2650 
-4850 
-5650 
-7950 
-1550 
-1000 
-4100 
-5000 
-9650 
-5450 


21.2450 
20.0500 
19.9550 
20.4550 
19.4050 
18.0350 
17.7500 
16.6400 
17.3150 
18.0350 
15.9600 
16.3050 
15.3900 
15.0150 
15.8450 
16.6000 
18.4400 
18.7850 
20.1150 
22.4900 
23.1800 
24.4850 
21.8150 
22.3500 
21.5800 
20.6700 
18.0700 
19.6450 
17 .5800 
16.4550 
16.1900 
16.3900 
15.4100 
15.0900 


591 


This bivariate series consists of measurements of luteinizing hormone (LH) 
at 10-minute intervals for two different cows (labeled cows 2 and 3 in the original 
study (see Rahe, et al. (1980)) each at day three of their estrous cycle. See 
Problem C3.23 for more discussion of this data set. 


Cow 2 on Day 3 of Cycle 


144 
14 


15 16 
13 20 
Li”yis 


14 
16 


14 20 
20 14 
19 14 
19° 17 
13 11 
13 11 
10 14 
ee 4 


592 DATA SETS APP. D. 


12 gids )9 10088) 8 eon 40 

Cow 3 on Day 3 of Cycle 
144 
14 11 11 16 13 12 10 9 20 16 13 13 11 9 16 15 13 
14 10 12 18 13 13 11 12 20 20 16 14 13 21 17 #14 #14 
12 19 18 16 15 14 16 20 17 14 15 12 18 18 15 13 13 
22 16 15 14 12 20 19 16 14 13 12 21 19 18 15 15 25 
21 17 14 18 19 14 14 14 19 19 17 14 13 21 17 16 14 
16 19 18 14 13 18 18 15 16 12 19 146 14 12 18 18 15 
16 12. 14 17.15 13.41 20 17 14 13 14 14 14 16 13 16 
17 p15) 13) 1194651946) 43 taeedei7, 16 15 14 19 19 15 
13 51S 11 ieee fee 15 Sed9 


New York City Temperatures and Births: NYCTB.DAT 


This bivariate series consists of average temperatures and the number of 
babies delivered in New York City for the 168 months from 1946 to 1959. 


HYC Monthly Temperatures 1946-59 

168 
11.506 11.022 14.405 14.442 16.524 17.918 18.959 18.309 
18.160 16.691 14.480 17.862 12.082 10.558 12.138 14.442 
16.152 17.714 19.015 19.164 17.900 16.933 13.364 11.468 
10.000 10.985 12.993 14.480 16.134 17.862 19.238 19.089 
18.067 15.576 14.926 12.398 12.435 12.416 13.067 15.093 
16.766 18.680 19.796 19.424 17.398 16.877 13.866 12.658 
13.048 11.301 12.063 14.164 16.041 18.067 19.108 18.736 
17.212 16.394 13.922 11.691 11.970 11.970 12.900 14.888 
16.747 17.955 19.201 18.903 17.714 16.097 13.234 12.435 
11.952 11.970 12.639 15.204 16.283 18.699 19.851 18.959 
18.030 15.316 14.126 12.323 12.230 12.230 13.234 14.647 
16.729 18.494 19.312 19.164 18.011 16.338 14.368 12.751 
11.022 12.658 12.844 15.037 16.190 18.253 19.182 18.513 
17.621 16.506 13.717 11.840 10.855 11.636 12.825 15.019 
16.952 17.937 20.000 19.535 17.770 16.283 13.327 10.818 
11.264 12.026 12.230 14.126 15.818 18.420 18.699 18.903 
17.175 16.041 13.866 12.844 10.613 12.082 12.955 15.000 
16.747 18.792 19.312 18.680 18.067 15.632 14.349 12.732 
11.152 10.372 12.639 14.981 16.097 17.602 19.331 18.978 
17.695 15.539 14.182 10.781 11.059 11.152 12.621 14.981 
17.100 18.160 19.015 19.331 18.346 16.115 13.532 12.398 

HYC Monthly Births 1946-1959 
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26.663 23.598 26.931 24.740 25.806 24.364 24.477 23.901 
23.175 23.227 21.672 21.870 21.439 21.089 23.709 21.669 
21.752 20.761 23.479 23.824 23.105 23.110 21.759 22.073 
21.937 20.035 23.590 21.672 22.222 22.123 23.950 23.504 
22.238 23.142 21.059 21.573 21.548 20.000 22.424 20.615 
21.761 22.874 24.104 23.748 23.262 22.907 21.519 22.025 
22.604 20.894 24.677 23.673 25.320 23.583 24.671 24.454 
24.122 24.252 22.084 22.991 23.287 23.049 25.076 24.037 
24.430 24.667 26.451 25.618 25.014 25.110 22.964 23.981 
23.798 22.270 24.775 22.646 23.988 24.737 26.276 25.816 
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DATA SETS 


24.707 
26 .606 
26.199 
25 .688 
28.599 
26.975 
26 .987 
28.141 
26.589 
26.619 
30.000 


Waves Data: WAVES.DAT 


24.364 
26.268 
27.210 
24.990 
27.914 
28.527 
26.589 
29.048 
27 .931 
26.076 
29.261 


22.644 
26.462 
26.122 
24.239 
27 .784 
27.139 
24.848 
28.484 
28.009 
25.286 
29.012 


This data set is discussed in Example 3.22. 
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APPENDIX E 


Index of Commands, Theorems, 
Macros, Examples, and Figures 


E.1. Commands 


All of the TIMESLAB commands are described in Appendix C, while most 
of them are discussed somewhere else in the book. The page numbers in this 
list refer to the best description of the use of the commands. 


ABORTOFF Henceforth Macros Don’t Abort upon an Error 477 
ABORTON Henceforth Macros Do Abort upon an Error 477 
ABS Absolute Value Command AT7 
ARCORR Find Correlations of Univariate AR Process 127 
ARCORR2 Find Correlations of Bivariate AR Process 324 
ARDT Difference Equation Command or Simulate AR Data 128 
ARDT2 Bivariate Version of ARDT 325 
ARFILT Apply AR Filter 129 
ARMACORR Find Correlations of ARMA Process 141 
ARMADT Simulate ARMA Data 483 
ARMAPRED Find ARMA Predictors 134 
ARMASEL Stepwise ARMA Modeling 226 
ARMASP Find Spectral Density of ARMA Process 487 
ARPART Transform AR Coefficients to Partials 126 
ARSP Find Spectral Density of AR Process 489 
ARSP2 Find Spectral Density of Bivariate AR Process 326 
ARSPCB Find Confidence Bands for an AR Spectral Density 253 
ARSPPEAK Estimate Peak Frequencies of an AR Spectral Density 259 
BARTTEST Bartlett’s Test for White Noise ifs 
BATCHOFF Cancel BATCHON 493 
BATCHON Henceforth No Pauses after Plots 494 
BINOM Find Binomial Coefficients or Probabilities 494 
CLEAN Delete Variables 495 
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CLS 
COEFFCSD 
COLOR 
CORR 
CORR2 
CORRAR 
CORRAR2 


CORRARMA 


CORRMA 
COs 
CROSSP 
CUM 
CUMSP 
DELAY 
DENSITY 
DIFF 
DIST 
DIVSDS 
DOS 

DOT 
DOUBLE 
DTAR 
DTARMA 
DTFORE 
ECHO 
EDIT 

EIG 

END 
ENDIF 
ERASE 
EXP 
EXTEND 
EXTRACT 
FFT 

FILT 
GOTO 
GRMENU 


INFQNT 
INVPOLY 
LABEL 


COMMANDS, THEOREMS, MACROS, EXAMPLES, FIGURES 


Clear the Screen 

Standard Errors for ARMA Parameter Estimates 
Set Graphics Colors 

Find Sample Correlogram and/or Periodogram 
Estimate Auto- and Cross-Correlations from Data 
Find AR Parameters from Correlations 

Find Bivariate AR Parameters from Correlations 
Find ARMA Parameters from Correlations 

Find MA Parameters from Correlations 

Find Cosine or Generate Points on a Cosine Curve 
Find Window Cross-Spectral Estimates 

Find Cumulative Sums or Averages of an Array 
Find Cumulative Spectral Density 

Make TIMESLAB Pause for a Specified Time 
Nonparametric Probability Density Estimation 
Difference an Array 

Find pdf, cdf, or Quantiles of Z, t, x”, or F 

Find Seasonal Standard Deviations 

Issue DOS Commands from TIMESLAB 

Find the Inner Product of Two Arrays 

Extend a Cross-Spectral Array by Symmetry 

Fit AR Models to Univariate Data 

Fit ARMA Models from Data 

Iterated AR Forecasting 

Henceforth Text on the Screen Echoed to Printer 
Invoke the TIMESLAB Text Editor 

Find Eigenvalues and Eigenvectors 

Terminator of WHILE...END Construct in Macros 
Terminator of IF...ENDIF Construct in Macros 
Erase Part of a Graphics Screen 

Exponentiation Command 

Extend an Array by Inverse Differencing 

Extract Part of an Array 

Fast Fourier Transform Command 

General Linear Filter Command 

Unconditional Branch in Macros 

Place the Graphics Menu onto a Graphics Screen 
Gram-Schmidt Decomposition of a Matrix 
On-line Help System 

Plot Histogram of an Array 

Conditional Branch in Macros 

Display a List of All Defined Variables 

Plot Informative Quantile Function of an Array 
Find Coefficients of the Reciprocal of a Polynomial 
Define a New Label or Plot a String of Characters 
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LENGTH List the Number of Elements of an Array 533 
LINE Form Values That Fall on a Line 533 
LIST Display the Values of an Array or Set of Scalars 534 
LISTM List a Two- or Three-Dimensional Array 535 
LISTSP List the Values of a Spectral Array 535 
LOGE Natural Logarithm Command 536 
MACORR Find Correlations of an MA Process 536 
MACRO Invoke a Macro or Reinvoke an Interrupted Macro 537 
MADT Simulate Data from Gaussian MA Process 538 
MASP Find Spectral Density of MA Process 538 
MAXMIN Find an Array’s Maximum and Minimum 539 
MCHOL Modified Cholesky Decomposition of a Matrix 359 
MDEL Delete Some Rows of a Matrix 540 
MINV Invert a Matrix 541 
MMULT Multiply Matrices 541 
MULTPOLY Find Coefficients of the Product of Polynomials 542 
NOTES User-Definable Help System 543 
OVEROFF TIMESLAB Will Ask Permission to Overwrite 543 
OVERON TIMESLAB Automatically Overwrites 943 
PAGE Reset Printer to Top of Next Page 544 
PARCORR Calculate Partials and Residual Variances of Data 10 
PARTAR Find Partial Correlations from AR Coefficients 126 
PAUSE Make a Macro Pause and Wait for a Response 545 
PLOT Y Versus X plot 546 
PLOT2 Superimpose Two Y Versus X Plots 547 
PLOTCSP Plot Spectral Array on Nonlog Scale 21 
PLOTK Superimpose K Y Versus X Plots 548 
PLOTOFF Switch from Graphics to Text Mode 549 
PLOTON Switch from Text to Graphics Mode 550 
PLOTSIZE Set Graphics Characteristics 550 
PLOTSP Plot Spectral Array on Log Scale 21 
POLAR Find Modulus and Arctangent of Two Arrays 307 
POLY Evaluate the Values of a Polynomial 553 
POLYROOTS Find Roots of a Polynomial 554 
PRINT Send a List of Variables to Printer 554 
PRINTER Set Printer Characteristics 555 
PRINTSEL Specify Printer Type 556 
PROMPTOFF Henceforth Each Command in Macro Not Displayed 556 
PROMPTON Henceforth Each Command in a Macro Displayed 557 
PSOFF Stop Automatic Printing of Graphs 557 
PSON Henceforth All Plots Also Printed 557 
QTEST Q Test for White Noise 174 
QUIT Exit from TIMESLAB and Return to DOS 558 
READ Read an Array from a File 558 
RECORD Keep a Record of a Session onto a File 559 
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REG 
REPLACE 
RESCREEN 
RESTART 
REVERSE 
ROOTSPOLY 
SAVE 
SAVESC 
SEASEST 
SEASPRED 
SIN 
SINGLEOFF 
SINGLEON 
SORT 
SPEAKER 
SPEAKEROFF 
SPEAKERON 
SUBMNS 
SWEEP 
TEXTCOLOR 
TIME 

TOEPL 
TRANS 
TYPE2 
TYPE4 
WHILE 
WINDOW 
WN 


E.2. Theorems 


Theorem 1.4.1. 
Theorem 1.4.2. 
Theorem 1.4.3. 
Theorem 1.4.4. 
Theorem 1.6.1. 
Theorem 2.2.1. 
Theorem 2.3.1. 
Theorem 2.4.1. 
Theorem 2.5.1. 
Theorem 2.5.2. 
Theorem 2.5.3. 
Theorem 2.5.4. 
Theorem 2.5.5. 
Theorem 2.5.6. 


COMMANDS, THEOREMS, MACROS, EXAMPLES, FIGURES 


General Purpose Regression Command 

Replace Specified Elements of an Array 

Redisplay a Graphics Screen That Has Been Saved 
Reset Parameters and Restart TIMESLAB 
Reverse the Elements of an Array 

Find Coefficients of a Polynomial from Roots 

Save an Array to a File 

Save a Graphics Screen to a File 

Multiplicative Subset ARMA Estimation 
Multiplicative Subset ARMA Prediction 

Find Sine or Generate Points on a Sine Curve 
Stop Single-Step Execution of a Macro 

Start Single-Step Execution of a Macro 

Sort an Array 

Make the Speaker Sound 

Stop Speaker Beeping at Each Command in a Macro 
Make Speaker Beep at Each Command in a Macro 
Calculate Seasonal Means 

Matrix Sweep Command 

Set Colors in Text Mode 

Find Number of Seconds since Midnight 

Form Symmetric Toeplitz Matrix 

Find Transpose of a Matrix 

Assignment and Arithmetic Operations 

Define the Elements of or Concatenate Arrays 
Start a WHILE...END Construct 

Univariate Nonparametric Spectral Estimation 
Generate White Noise Data 


Orthogonality of Sinusoids 

Sinusoidal Decomposition of Data 
Standardizing the Periodogram 
Relations among /, R, f 

Solution of Difference Equations 

Two Spectral Representations 
Univariate Filter Theorem 

Univariate Prediction 

Properties of Random Walks 
Invertibility of MA Processes 
Properties of Autoregressive Processes 
Exponential Decay of AR Correlations 
Autoregressive Approximation 

Rand f for an ARMA Process 


101 
104 
105 
109 
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Theorem 2.5.7. 
Theorem 2.5.8. 
Theorem 2.6.1. 
Theorem 2.6.2. 
Theorem 2.6.3. 
Theorem 2.6.4. 
Theorem 2.6.5. 
Theorem 2.6.6. 
Theorem 2.6.7. 
Theorem 2.6.8. 
Theorem 3.1.1. 
Theorem 3.1.2. 
Theorem 3.1.3. 
Theorem 3.1.4. 
Theorem 3.3.1. 
Theorem 3.4.1. 
Theorem 3.4.2. 
Theorem 3.4.3. 
Theorem 3.4.4. 
Theorem 3.4.5. 
Theorem 3.4.6. 
Theorem 3.5.1. 
Theorem 3.5.2. 
Theorem 3.8.1. 
Theorem 3.8.2. 
Theorem 3.8.3. 
Theorem 3.8.4. 
Theorem 3.9.1. 
Theorem 3.9.2. 
Theorem 4.1.1. 
Theorem 4.1.2. 
Theorem 4.1.3, 
Theorem 4.1.4. 
Theorem 4.1.5. 
Theorem 4.1.6. 
Theorem 4.1.7. 
Theorem 4.1.8. 
Theorem 4.2.1. 
Theorem 4.2.2. 
Theorem 4.2.3. 
Theorem 4.2.4. 
Theorem 4.2.5. 
Theorem A.1.1, 
Theorem A.1.2. 
Theorem A.1.3. 


Inverse Parameters for ARMA Models 


Properties of the General Linear Process 


Prediction Via the MCD 
Gaussian Likelihood Function 
The Levinson Recursion 

Partial Autocorrelations 

The MCD of T, for an AR(p) 
The Schur Test for Stationarity 
ARMA Prediction Via the MCD 


ARMA State and Observation Equations 


Sampling Properties of X 

Positive Definiteness of R 

Properties of R and p 

Properties of the Periodogram 
Properties of the Window Estimators 
Properties of MLE’s for Time Series 


Asymptotic Distribution of ARMA MLE’s 


Properties of YWE’s 

Cybenko’s Theorem 

A “Given Data” Levinson Algorithm 
Properties of Partial Autocorrelations 
Inconsistency of AIC for an AR(p) 
Consistent AR Order Determination 
Conditions for AR(oo) Representation 
Properties of CAT 

Properties of AR Spectral Estimation 
Confidence Bands for AR Spectra 
Fisher’s Exact Test 

Estimating a Peak Frequency 
Spectral Representation of R 

Range of Coherence Spectrum 

Two Properties of Coherence 
Bivariate Filter Theorem 

Optimal Filters 

Approximating Filter Coefficients 
Stationarity of AR 

Bivariate Levinson Algorithm 
Properties of X and R 

Testing for Independence 

Properties of Bivariate Periodogram 
Bivariate Window Estimators 
Properties of Spectral Quantities 
Modified Cholesky Decomposition 
Gram-Schmidt Decomposition 
Properties of the Sweep Operator 
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Theorem A.1.4. 
Theorem A.2.1. 
Theorem A.2.2. 
Theorem A.2.3. 
Theorem A.2.4. 
Theorem A.3.1. 
Theorem A.4.1. 
Theorem A.4.2. 
Theorem A.4.3. 
Theorem A.4.4. 
Theorem A.4.5. 
Theorem A.4.6. 
Theorem A.5.1. 
Theorem A.5.2. 
Theorem A.5.3. 
Theorem A.5.4. 
Theorem A.5.5. 


E.3. Macros 


AIC 
AIREST 
ARGAMI 
ARMAAIC 
ARSP 
ARSPSM 
ARTREG 
AVEPER 
BAND 
BAR 
BARTCDF 
BAUER 
BINARY 
BURGYW 
CATWN 
CHIPLOT 
CIRCLE 
CIS 
COHER 
COHTEST 
CORR2PLT 
COs 
COSDE 
CROSSP 
CROSSPE 


COMMANDS, THEOREMS, MACROS, EXAMPLES, FIGURES 


The Matrix Inversion Lemma 

Fourier Series Approximation 
Properties of Fourier Series 

Spectral Representation of R 
Lebesgue-Stieltjes Integral 

The FFT Recursion 

Mean and Variance of a Linear Function 
Properties of the Multivariate Normal 
Simultaneous Confidence Intervals 
General Prediction Theory 

The Kalman Filter 

Continuous Function Theorem 
Properties of Least Squares Estimators 
Inferences for Normal Errors 

Central Limit Theorem for Regression 
Generalized Least Squares 

Recursive Regression Formulas 


AIC for AR Processes 

Parameter Estimates for Airline Data 

Inverse of Covariance Matrix for AR Process 
AIC for ARMA Process 

Autoregressive Spectral Estimation 

Illustrate Properties of AR Spectral Estimation 
Regression Residuals for Artificial Data 
Average Periodogram over a Frequency Band 
Bandpass Filter Example 

Bivariate Autoregressive Modeling 

Find CDF of Bartlett Statistic 

Illustrate Bauer’s Algorithm 

Form a Binary Series by Clipping 

Compare Burg and Yule-Walker Estimates 
Significance Level of CAT As White Noise Test 
Plots of y? pdf’s 

Use PLOT2 to Draw a Circle 

Confidence Intervals for the Mean of an AR(1) 
Illustrate Coherence Spectrum 

Test for Zero Coherence 

List and Plot Auto- and Cross-Correlations 
Illustrate Adding Cosines 

Generate a Cosine Curve Via Difference Equation 
Calculate Bivariate Cross-Spectra 

Estimate Bivariate Cross-Spectra 
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DESCRIBE Display Univariate Descriptive Statistics 47 
DIRICH Display Dirichlet Kernel 146 
DTFORE Forecasting Via Iterated AR Models 245 
EIG Illustrate Relationship of Eigenvalues and Spectra 143 
EXAMPLE Skeleton for General Macros 448 
EXPSM Choose Exponential Smoothing Parameter 57 
FEJER Display Fejér Kernel 266 
FIG41 Construct Figure 4.1 345 
FILTEST Estimate Coefficients of Optimal Filter 344 
FILTEX Illustrate Approximating Filter Coefficients 321 
FINDFILT Approximate Filter Coefficients 320 
FISHER Fisher’s Exact Test for Periodicity 286 
FISHPV p-Value for Fisher’s Exact Test 286 
FPLOT Plot F pdf’s 466 
GPAC Calculate and Display GPAC Array 217 
HAT2 Plot Rows of Hat Matrix for Quadratic Regression 462 
ID ARMA Model Identification 279 
IDARMA Illustrate Patterns in ARMA Quantities 150 
INCONS Illustrate Inconsistency of Periodogram 266 
INDTEST Test for Independence of Two Series 349 
INNOV Estimate Innovation Variance 273 
LAMBDA Finding Power Transform for Sales Data 233 
LREGAR Regression with AR Errors for Lynx Data 242 
LSAR Least Squares Estimates for AR Process 213 
LYNXCB AR Spectral Confidence Bands for Lynx Data 287 
LYNXPEAK Estimate Peak Frequency for Lynx Data 288 
MEDMN Example of Simulation Study Macro 457 
MISSING Fill in Missing Data 52 
MSERHO Compare Biased and Unbiased Sample Correlations 262 
NO1TAB Form Table of N (0,1) cdf 455 
ODE Illustrate the SPEAKER Command 460 
PARZ Calculate Parzen Lag Weights 148 
PERAVE Illustrate Interpretation of Spectral Density 145 
PLAY “Play” an Array on the Speaker 460 
PREWHITE Prewhiten and Align a Pair of Series 349 
PVH Calculate PVH 275 
QLEV Driver Macro for Effect of m on Q Test 268 
QLEVEL Effect of m in Q Test 269 
RANDCOEF Randomly Generate Coefficients of a Polynomial 149 
REG Ordinary Least Squares Regression 400 
REGAR Regression with AR Errors 241 
RW Generate and Display Random Walks 148 
SALESPRD Predict Sales Data 239 
SALESRHO Find Correlations for Sales Data Models 236 
SAWTOOTH Generate Values on a Sawtooth Function 54 
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SCATTER 
SIMFILT 
SLOW 
SMOOTH 
STDRHO 
TIMECOV 
WINDOW 
WINDSP 
WINDSP3 
WN 
WNTEST 


E.4. Examples 


Example 1.1. 
Example 1.2. 
Example 1.3. 
Example 1.4. 
Example 1.5. 
Example 1.6. 
Example 1.7. 
Example 1.8. 
Example 1.9. 
Example 1.10. 
Example 1.11. 
Example 1.12. 
Example 2.1. 
Example 2.2. 
Example 2.3. 
Example 2.4. 
Example 2.5. 
Example 2.6. 
Example 2.7. 
Example 2.8. 
Example 3.1. 
Example 3.2. 
Example 3.3. 
Example 3.4. 
Example 3.5. 
Example 3.6. 
Example 3.7. 
Example 3.8. 
Example 3.9. 
Example 3.10. 
Example 3.11. 


COMMANDS, THEOREMS, MACROS, EXAMPLES, FIGURES 


Form Scatterplot for Data Versus Lagged Data 
Simulate Data from Transfer Function Model 
Organizing Calculations to Save Time 
Example of Moving Average Smoother 

Find Standard Error of Correlogram 

Compare Two Methods of Finding Correlations 
Display Spectral Windows 

Window Estimate and Confidence Intervals 
Window Estimate for Three Truncation Points 
The Eight Distributions in the WN Command 
Test for White Noise 


White Noise Data 

Descriptive Statistics 

Scatterplots 

Sum of Four Cosines 

Testing for White Noise 

Missing Data 

Binary Time Series 

The Sawtooth Function 

Two Methods of Finding Correlations 
Residuals from Regression 
Moving Average Smoother 
Exponential Smoothing 
Eigenvalues of Toeplitz Matrices 
The Spectral Density Function 
The Dirichlet Kernel 

The Bandpass Filter Example 
Random Walks 

Generating Stable Coefficients 
Patterns in ARMA Processes 
Bauer’s Algorithm 

Confidence Intervals for the Mean 
MSE for f and p 

Standard Error of Autocorrelations 
Inconsistency of the Periodogram 
The Fejér Kernel 

CDF of the Bartlett Statistic 

The Choice of m in the Q Test 
Averaging the Periodogram 
Spectral Windows 

Window Estimators 

Estimating the Innovation Variance 
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Example 3.12. 
Example 3.13. 
Example 3.14. 
Example 3.15. 
Example 3.16. 
Example 3.17. 
Example 3.18. 
Example 3.19. 
Example 3.20. 
Example 3.21. 
Example 3.22. 


Example 4.1. 
Example 4.2. 
Example 4.3. 
Example 4.4. 
Example 4.5. 
Example 4.6. 
Example B.1. 
Example B.2. 
Example B.3. 
Example B.4. 
Example B.5. 
Example B.6. 


E.5. Figures 


Figure 1.1. 
Figure 1.2. 
Figure 1.3. 
Figure 1.4. 
Figure 1.5. 
Figure 1.6. 
Figure 1.7. 
Figure 1.8. 
Figure 1.9. 
Figure 1.10. 
Figure 1.11. 
Figure 1.12. 
Figure 1.13. 
Figure 1.14. 
Figure 1.15. 
Figure 1.16. 
Figure 2.1. 
Figure 2.2. 
Figure 2.3. 


Yule-Walker and Burg Estimators 
Prediction Variance Horizons 

CAT As a White Noise Test 

AIC for ARMA Processes 

Model Identification 

Fisher’s Exact Test 

Confidence Bands for AR Spectra 
Estimating Peak Frequencies 
Autoregressive Spectral Estimation 
Properties of AR Spectral Estimation 
Autoregressive Filtering 

Forming Table 4.1 and Figures 4.1 and 4.2 
Interpreting Coherence 

Testing for Independence 
Nonparametric Spectral Estimation 
Testing for Zero Coherence 

Bivariate Autoregressive Modeling 
Forming Tables 

A Simulation Study 

Arranging Calculations to Save Time 
Using the Speaker 

Drawing a Circle 

Superimposing Plots 


Series I-X 

Scatterplots 

Correlograms for Series I-X 

Partials and Residual Variances for Series I-IX 
Sums of Cosines and Their Periodograms 
Periodograms for Series I-IX 

Cumulative Periodograms for Series I-IX 
Sample Spectral Density for Series IX 
Residuals from Regression for Series V 
Example of Moving Average Smoother 

Airline Data and Extended Values 

The Distribution of Differenced Wheat Price Data 
Example Output from DESCRIBE.MAC 

Example Output from WNTEST.MAC 

Two Binary Series 

The Sawtooth Function 

Wolfer Sunspot Data 

Interpreting the Spectral Density Function 
The Dirichlet Kernel 
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Figure 2.4. 
Figure 2.5. 
Figure 2.6. 
Figure 2.7. 
Figure 2.8. 
Figure 2.9. 
Figure 3.1. 
Figure 3.2. 
Figure 3.3. 
Figure 3.4. 
Figure 3.5. 
Figure 3.6. 
Figure 3.7. 
Figure 3.8. 
Figure 3.9. 


Figure 3.10. 
Figure 3.11. 
Figure 3.12. 
Figure 3.13. 
Figure 3.14. 
Figure 3.15. 
Figure 3.16. 
Figure 3.17. 
Figure 3.18. 
Figure 3.19. 


Figure 4.1. 
Figure 4.2. 
Figure 4.3. 
Figure 4.4. 
Figure 4.5. 
Figure 4.6. 
Figure 4.7. 
Figure B.1. 
Figure B.2. 
Figure B.3. 
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Fourier Series Approximation 

Five Realizations from a Random Walk Process 
Examples of Moving Average Processes 
Examples of Autoregressive Processes 
Examples of ARMA Processes 

Airline Data and Transformations 
Confidence Intervals for the Mean 

The Féjer Kernel 

The Inconsistency of the Periodogram 
The cdf of the Bartlett Statistic 
Averaging the Periodogram 

Eight Spectral Windows 

The Parzen and Tukey Spectral Windows 


Parzen Window Spectral Estimates for Three Series 


Prediction Variance Horizon for Airline Data 
Identifying Power Transformations 

Sales Data and Box-Jenkins Forecasts 

Sales Data and Iterated AR Forecasts 

AR Spectral Estimates for Three Series 

AR Spectral Confidence Bands for Lynx Data 
Lynx and Sunspot Data 

AR Spectral Estimate for Log Lynx Data 


Correlogram and Confidence Bands for Rainfall Data 


Yule-Walker and Burg Spectral Estimates 
The Effect of Autoregressive Filtering 


Cross-Correlations and Data from a Bivariate Process 


Auto- and Cross-Spectra for a Bivariate Process 
Interpreting Coherence Spectra 
Cross-Correlations and Confidence Bands 
Coherence and Phase and Confidence Intervals 
Testing for Zero Coherence 

Phase and Coherence for Gas Furnace Data 
Using Scaling Information in Plotting Commands 
Displaying the Output of a Simulation Study 
Superimposing Graphs 


REFERENCES 


Ahlfors, L.V. (1953). “Complex Analysis.” McGraw-Hill, New York. 


Akaike, H. (1971). Information theory and an extension of the maximum like- 
lihood principle. Research Memorandum No. 46, Institute of Statistical 
Mathematics, Tokyo. Published in 2nd Int. Symp. on Inf. Theory, ed. by 
B.N. Petrov and F. Csaki, Akademiai Kiade, Budapest (1973). 


Anderson, T.W. (1971). “The Statistical Analysis of Time Series.” Wiley, New 
York. 


Anderson, T.W. (1975). Maximum likelihood estimation of parameters of an 
autoregressive process with moving average residuals and other covariance 
matrices with linear structure. Ann. Statist., 3, 1283-1304. 


Ansley, C.F. (1979). An algorithm for the exact likelihood of a mixed autore- 
gressive moving average process. Biometrika, 66, 59-65. 


Ansley, C.F., and Kohn, R. (1985). Estimation, filtermg, and smoothing in 
state space models with incompletely specified initial conditions. Ann. 
Statist., 13, 1286-1316. 


Bartlett, M.S. (1950). Periodogram analysis and continuous spectra. 
Biometrika, 37, 1-16. 


Bartlett, M.S. ( 1955). “An Introduction to Stochastic Processes with Special 
Reference to Methods and Applications.” Cambridge University Press, 


London. 


Bauer, F.L. (1955). Ein direktes iterationsverfahren zur Hurwitz zerlegung 
eines polynoms. Archiv. Elekt. Ubertr., 9, 285-290. 


604 


REFERENCES 605 


Berk, K.N. (1974). Consistent autoregressive spectral estimates. Ann. Statist., 
2, 489-502. 


Bhansali, R.J. (1986). Asymptotically efficient selection of the order by the 
criterion autoregressive transfer function. Ann. Statist., 14, 315-325. 


Bloomfield, P. (1976). “Fourier Analysis of Time Series: An Introduction.” 
Wiley, New York. 


Bohman, H. (1960). Approximate Fourier analysis of distribution functions. 
Ark. Mat., 4, 99-157. 


Box, G.E.P., and Cox, D.R. (1964). An analysis of transformations. J. Roy. 
Statist. Soc. Ser. B, 26, 211-243. 


Box, G.E.P., and Jenkins, G.M. (1970). “Time Series Analysis, Forecasting, 
and Control.” Holden-Day, San Francisco. 


Box, G.E.P., and Jenkins, G.M. (1973). Some comments on a paper by Chat- 
field and Prothero and on a review by Kendall. J. Roy. Statist. Soc. Ser. 
A, 135, 337-345. 


Box, G.E.P., and Pierce, D.A. (1970). Distribution of residual autocorrelations 
in autoregressive-integrated moving average time series models. J. Amer. 
Statist. Assoc., 65, 1509-1526. 


Box, G.E.P., and Tiao, G.C. (1975). Intervention analysis with applications 


to economic and environmental problems. J. Amer. Statist. Assoc., 70, 
70-79. 


Brigham, E.O. (1974). “The Fast Fourier Transform.” Prentice-Hall, Engle- 
wood Cliffs, N.J. 


Brillinger, D.R. (1975). “Time Series Analysis: Data Analysis and Theory.” 
Holt, Rinehart & Winston, New York. 


Brockwell, P.J., and Davis, R.A. (1985). Applications of a recursive prediction 
algorithm in time series analysis. Statistics Department, Colorado State 
University. 


Brockwell, P.J., and Davis, R.A. (1987). “Time Series: Theory and Methods.” 
Springer-Verlag, New York. 


Burg, J.P. (1967). Maximum entropy spectral analysis. 37th Annual Interna- 
tional S.E.G. Meeting, Oklahoma City, Oklahoma. 


Chatfield, C., and Prothero, D.L. (1973). Box-Jenkins seasonal forecasting: 
problems in a case study. J. Roy. Statist. Soc. Ser. A, 136, 295-336. 


606 REFERENCES 


Clayton, D.G. (1971). Gram-Schmidt orthogonalization. Appl. Statist., 20, 
335-338. 


Cleveland, W.S. (1972). The inverse autocorrelations of a time series and their 
applications. Technometrics, 14, 277-293. 


Cogburn, R., and Davis, H.T. (1974). Periodic splines and spectral estimation. 
Ann. Statist., 2, 1108-1126. 


Cooley, J.W., and Tukey, J.W. (1965). An algorithm for the machine calcula- 
tion of complex Fourier series. Math. Comp., 19, 297-301. 


Cybenko, G. (1983). A general orthogonalization technique with applications 
to time series analysis and signal processing. Math. Comp., 40, 323-336. 


Davis, H.T. (1980). Comments on Pagano’s “Some recent advances in time 
series,” in “Directions in Time Series,” ed. by D.R. Brillinger and G.C. 
Tiao, Institute of Mathematical Statistics, Hayward, California, 301-302. 


Davis, H.T., and Jones, R.H. (1968). Estimation of the innovation variance of 
a stationary time series. J. Amer. Statist. Assoc., 63, 141-149. 


DeMeersman, R. (1975). A method for least squares solution of systems with 
a cyclic rectangular coefficient matrix. J. Comput. Appl. Math., 1, 51-54. 


Draper, N.R., and Smith, H. (1981). “Applied Regression Analysis.” Wiley, 
New York. 


Durbin, J. (1959). Efficient estimation of parameters in moving average models. 
Biometrika, 46, 306-316. 


Durbin, J. (1960). The fitting of time series models. Rev. Inst. Internat. 
Statist., 28, 233-244. 


Dzhaparidze, K. (1986). “Parameter Estimation and Hypothesis Testing in 
Spectral Analysis of Stationary Time Series.” Springer-Verlag, New York. 


Engel, E.M.R. (1984). A unified approach to the study of sums, products, 
time-aggregation and other functions of ARMA processes. J. Time Series 
Anal., 5, 159-171. 


Ensor, K. (1987). PhD Dissertation, Dept. of Statistics, Texas A&M Univer- 
sity. 


Ensor, K., and Newton, H.J. (1987). The effect of order determination on 
estimating the peak frequency of an autoregressive spectral density. De- 
partment of Statistics, Texas A&M University. 


REFERENCES 607 


Feller, W. (1948). On the Kolmogarov-Smirnov theorems for empirical distri- 
butions. Ann. Math. Statist., 19, 177-189. 


Findley, D.F. (1978). Limiting autocorrelations and their uses in the identi- 
fication of nonstationary ARMA models. Bull. Inst. Math. Statist., 7, 
293-294. 


Fisher, R.A. (1929). Tests of significance in harmonic analysis. Proc. Roy. 
Soc. Ser. A, 129, 54-59. 


Fishman, G.S. (1973). “Concepts and Methods in Discrete Event Digital Sim- 
ulation.” Wiley, New York. 


Friedlander, B., Morf, M., Kailath, T., and Ljung, L. (1979). New inversion 
formulas for matrices classified in terms of their distance from Toeplitz 
matrices. Lin. Alg. Appl., 60, 31-60. 


Fuller, W.A. (1976). “Introduction to Statistical Time Series.” Wiley, New 
York. 


Gentleman, W.M., and Sande, G. (1966). Fast Fourier transforms-for fun and 
profit. AFIPS, 1966 Fall Joint Computer Conference, Conf. Proc., 28, 
563-578. 


Granger, C.W.J., and Morris, M.J. (1976). Time series modeling and interpre- 
tation. J. Roy. Statist. Soc. Ser. A, 139, 246-257. 


Gray, H.L., and Woodward, W. (1986). A new ARMA spectral estimator. J. 
Amer. Statist. Assoc., 81, 1100-1108. 


Grenander, U., and Rosenblatt, M. (1957). “Statistical Analysis of Stationary 
Time Series.” Wiley, New York. 


Grenander, U., and Szego, G. (1958). “Toeplitz Forms and Their Applica- 
tions.” Univ. of California Press, Berkeley. 


Hannan, E.J. (1970). “Multiple Time Series.” Wiley, New York. 


Hannan, E.J., and Nicholls, D.F. (1977). The estimation of the prediction error 
variance. J. Amer. Statist. Assoc., 72, 834-840. 


Hannan, E.J., and Quinn, B.G. (1979). The determination of the order of an 
autoregression. J. Roy. Statist. Soc. Ser. B, 41, 190-195. 


Hannan, E.J., and Rissanen, J. (1982). Recursive estimation of mixed autore- 
gressive moving average order. Biometrika, 69, 81-94. 


Harrison, P.J., and Stevens, C.F. (1976). Bayesian forecasting. J. Roy. Statist. 
Soc. Ser. B, 38, 205-258. 


608 REFERENCES 


Hartley, H.O. (1949). Tests of significance in harmonic analysis. Biometrika, 
36, 194-201. 


Isserlis, L. (1918). On a formula for the product moment coefficient of any order 
of a normal frequency distribution in any number of variables. Biometrika, 
12, 134-139. 


Jenkins, G.M., and Watts, D.G. (1968). “Spectral Analysis and Its Applica- 
tions.” Holden-Day, San Francisco. 


Jennrich, R.I. (1977). Stepwise regression, in “Statistical Methods for Digital 
Computers,” ed. by K. Enslein, A. Ralston, and H.S. Wilf, Wiley, New 
York. 


Jones, R.H. (1980). Maximum likelihood fitting of ARMA models to time series 
with missing observations. Technometrics, 22, 389-395. 


Jones, R.H., and Brelsford, W.M. (1967). Time series with periodic structure. 
Biometrika, 54, 403-408. 


Kalman, R.E. (1960). A new approach to linear filtering and prediction prob- 
lems. Trans. ASME, J. Basic Eng., 83D, 35-45. 


Kedem, B. (1980). “Binary Time Series.” Marcel Dekker, New York. 


Kedem, B. (1986). Search for periodicities by axis crossings of filtered time 
series. Signal Processing, 10, 129-144. 


Levinson, N. (1947). The Wiener RMS error criterion in filter design and 
prediction. J. Math. Physics, 25, 261-278. 


Lewis, P.A.W., Goodman, A.S., and Miller, J.M. (1969). Pseudo-random num- 
ber generator for the System/360. IBM Systems J., 8, 136-146. 


Ljung, G.M., and Box, G.E.P. (1978). On a measure of lack of fit in time series 
models. Biometrika, 65, 297-303. 


Makhoul, J. (1975). Linear prediction: a tutorial review. Proceedings of the 
IEEE, 63, 561-580. 


Makhoul, J. (1977). Stable and efficient lattice methods for linear prediction. 
IEEE Trans. on Acoustics, Speech, and Signal Processing, ASSP-25, 
423-428. 


Makridakis, §., Andersen, A., Carbone, R., Fildes, R., Hibon, M., 
Lewandowski, R., Newton, J., Parzen, E., and Winkler, R. (1984). “The 
Forecasting Accuracy of Major Time Series Methods.” Wiley, New York. 


REFERENCES 609 


Marsaglia, G. (1962). Random variables and computers, Information The- 
ory Statistical Decision Functions Random Processes: Transactions of the 
Third Prague Conf., ed. by J. Kozesnik, Czechoslovak Academy of Sci- 
ences, Prague, 499-510. 


Masani, P. (1966). Recent trends in multivariate prediction theory, in Mul- 
tivariate Analysis—I, ed. by P. Khrishnaiah, Academic Press, New York, 
351-382. 


Meinhold, R.J., and Singpurwalla, N.D. (1983). Understanding the Kalman 
filter. Amer. Statist., 37, 123-127. 


Melard, G. (1984). A fast algorithm for the exact likelihood of autoregressive- 
moving average models. Appl. Statist., 33, 104-114. 


Morris, J. (1977). Forecasting the sunspot cycle. J. Roy. Statist. Soc. Ser. A, 
140, 437-447. 


Newton, H.J. (1982). Using periodic autoregressions for multiple spectral esti- 
mation. Technometrics, 24, 109-116. 


Newton, H.J., and Pagano, M. (1982). The finite memory prediction of a 
covariance stationary time series. SIAM J. Sct. Stat. Comp., 4, 330-339. 


Newton, H.J., and Pagano, M. (1983a). A method for determining periods in 
time series. J. Amer. Statist. Assoc., 78, 152-157. 


Newton, H.J., and Pagano, M. (1983b). Computing for autoregressions, Proc. 
of 15th Interface of Stat. and Comp., ed. by J. Gentle, North Holland, 
Amsterdam, 125-130. 


Newton, H.J., and Pagano, M. (1984). Simultaneous confidence bands for 
autoregressive spectra. Biometrika, 71, 197-202. 


O’Neill, R. (1971). Function minimization using a simplex procedure. Appl. 
Statist., 20, 338-345. 


Pagano, M. (1973). When is an autoregressive scheme stationary? Communi- 
cations in Statistics, 1, 533-544. 


Pagano, M. (1976). On the linear convergence of a covariance factorization 
algorithm. J. Assoc. Comp. Mach., 23, 310-316. 


Pagano, M. (1978). On periodic and multiple autoregressions. Ann. Statist., 
6, 1310-1317. 


Parzen, E. (1957a). On consistent estimates of the spectrum of a stationary 
time series. Ann. Math. Statist., 28, 329-348. 


610 REFERENCES 


Parzen, E. (1957b). On choosing an estimate of the spectral density function 
of a stationary time series. Ann. Math. Statist., 28, 921-932. 


Parzen, E. (1958). On asymptotically efficient consistent estimates of the spec- 
tral density function of a stationary time series. J. Roy. Statist. Soc. Ser. 
B, 20, 303-322. 


Parzen, B. (1961). An approach to time series analysis. Ann. Math. Statist., 
32, 951-989. 


Parzen, E. (1962a). On estimation of a probability density function and mode. 
Ann. Math. Statist., 32, 1065-1076. 


Parzen, E. (1962b). “Stochastic Processes,” Holden-Day, San Francisco. 


Parzen, E. (1963a). Notes on Fourier analysis and spectral windows. Technical 
Report No. 48, May 1963, Statistics Department, Stanford University. 
Reprinted in Time Series Analysis Papers, Holden-Day, San Francisco. 


Parzen, E. (1963b). On spectral analysis with missing observations and ampli- 
tude modulation. Sankha, Ser. A, 25, 383-392. 


Parzen, E. (1967). Empirical multiple time series analysis. Proc. of the Fifth 
Berkeley Symposium on Mathematical Statistics and Probability, ed. by L. 
LeCam and J. Neyman, University of California Press, Vol. I, 305-340. 


Parzen, E. (1971). Efficient estimation of stationary time series mixed schemes. 
Bull. Inst. Internat. Statist., 44, 315-319. 


Parzen, E. (1977). Multiple time series: Determining the order of approxi- 
mating autoregressive schemes. In Multivariate Analysis - IV, ed. by P. 
Khrishnaiah, North Holland, Amsterdam, 283-295. 


Parzen, E. (1979). Nonparametric statistical data modeling. J. Amer. Statist. 
Assoc., 74, 105-121. 


Parzen, E. (1981). Time series model identification and prediction variance 
horizon. Proc. of Second Tulsa Symposium on Applied Time Series Anal- 
ysis, Academic Press, New York, 415-447. 


Parzen, E. (1982). ARARMA models for time series analysis and forecasting. 
J. of Forecasting, 1, 67-82. 


Parzen, E. (1983a). Informative quantile functions and identification of prob- 
ability distribution types. Proc. of 29th Army Conference on the Design 
of Experiments, 97-107. 


REFERENCES 611 


Parzen, E. (1983b). “Time Series Analysis of Irregularly Observed Data,” Pro- 
ceedings of a Symposium held at Texas A&M University, College Station, 
Texas, February 10-13, 1983. 


Priestley, M.B. (1981). “Spectral Analysis and Time Series,” Vols. 1 and 2. 
Academic Press, New York. 


Pukkila, T., and Nyquist, H. (1985). On the frequency domain estimation 
of the innovation variance of a stationary time series. Biometrika, 72, 
$1 i323: 


Quinn, B.G. (1980). The limiting behavior of the autocorrelation function of 
ARMA processes as several roots of the characteristic equation approach 
the unit circle. Communications in Statistics, B, 9, 195-198. 


Rahe, C.H., Owens, R.E., Fleeger, J.L., Newton, H.J., and Harms, P.G. (1980). 
Pattern of plasma luteinizing hormone in the cyclic cow: dependence upon 


the period of the cycle. Endocrinology, 107, 498-530. 


Rao, C.R. (1973). “Linear Statistical Inference and Its Applications.” Wiley, 
New York. 


Scheffé, H. (1959). “The Analysis of Variance.” Wiley, New York. 


Serfling, R.J. (1980). “Approximation Theorems of Mathematical Statistics.” 
Wiley, New York. 


Shibata, R. (1976). Selection of the order of an autoregressive process by 
Akaike’s information criterion. Biometrika, 63, 117-126. 


Shibata, R. (1981). An optimal autoregressive spectral estimate. Ann. Statist., 
9, 300-306. 


Siddiqui, M.M. (1962). Some statistical theory for the analysis of radio pro- 
pogation data. J. of Research of Nat. Bur. Stds., 66D, 571-580. 


Silverman, B.W. (1986). “Density Estimation for Statistics and Data Analy- 
sis.” Chapman and Hall, New York. 


Singleton, R.C. (1969). An algorithm for computing the mixed radix fast 
Fourier transform. [EEE Trans. Audio Electron., AU-17, 93-103. 


Stewart, G.W. (1973). “Introduction to Matrix Computations.” Academic 
Press, New York. 


Stuart, A. (1955). A paradox in statistical estimation. Biometrika, 42, 527- 
529. 


612 REFERENCES 


Tapia, R.A., and Thompson, J.R. ( 1978). “Nonparametric Probability Density 
Estimation.” Johns Hopkins University Press, Baltimore. 


Tjgstheim, D., and Paulsen, J. (1983). Bias of some commonly-used time series 
estimates. Biometrika, 70, 389-399. 


Tong, H. (1977). Some comments on the Canadian lynx data. J. Roy. Statist. 
Soc. Ser. A, 140, 432-436. 


Tsay, R.S. (1986). Behavior of sample means of nearly nonstationary time 
series. Proc. of the 1986 Winter Simulation Conference, 351-355, Society 
for Computer Simulation, San Diego. 


Tsay, R.S., and Tiao, G.C. (1984). Consistent estimates of autoregressive 
parameters and extended sample autocorrelation functions for stationary 
and nonstationary ARMA models. J. Amer. Statist. Assoc., 79, 84-96. 


Tukey, J.W. (1949). The sampling theory of power spectrum estimates. Proc. 
Symp. on Applications of Autocorrelation Analysis to Physical Problems, 
NAVEXOS-P-735, 47-67. Office of Naval Research, Department of the 
Navy, Washington, D.C. 


Ulrych, T.L., and Bishop, T.N. (1975). Maximum entropy spectral analysis 
and autoregressive decomposition. Rev. Geo. Space Phys., 14, 183-200. 


Wahba, G. (1980). Automatic smoothing of the log periodogram. J. Amer. 
Statist. Assoc., 75, 122-132. 


Welch, P.D. (1967). The use of the fast Fourier transform for the estimation 
of power spectra: a method based on time averaging over short, modified 
periodograms. [EEE Trans. on Audio and Electroacoustics, AO-15, 70- 
73. 


Whittle, P. (1951). “Hypothesis Testing in Time Series Analysis.” Ph.D. 
Thesis, Uppsala. Almquist & Wiksell, Uppsala. 


Whittle, P. (1963a). “Prediction and Regulation.” English Universities Press, 
London. 


Whittle, P. (1963b). On the fitting of multivariate autoregressions, and the ap- 
proximate canonical factorization of a spectral density matrix. Biometrika, 
50, 129-134. 


Wilson, G. (1969). Factorisation of the covariance generating function of a 
pure moving average process. SIAM J. Numer. Anal., 6, 1-7. 


Wilson, G. (1979). Some efficient computational procedures for high order 
ARMA models. J. Statist. Comput. Simul., 8, 301-309. 


REFERENCES 613 


Woodroofe, M.B., and Van Ness, J.W. (1967). The maximum deviation of 
sample spectral densities. Ann. Math. Statist., 38, 1558-1570. 


Woodward, W.., and Gray, H.L. (1981). On the relationship between the S array 
and the Box-Jenkins method of model identification. J. Amer. Statist. 
Assoc., 76, 579-587. 


AUTHOR INDEX 


Abhlfors, L.V., 102 

Akaike, H., 220 

Andersen, A., 243 

Anderson, T.W., 104, 161, 166, 168, 
189, 213, 295, 363, 394, 580 

Ansley, C.F., 114, 129 


Bartlett, M.S., 167, 173, 298 

Bauer, F.L., 135 

Berk, K.N., 248 

Bhansali, R.J5 223 

Bishop, T.N., 211 

Bloomfield, Paai2 

Bohman, H., 184 

Box, G.E.P., 113, 174, 200, 202, 211, 
230, 232, 233, 243, 343, 344, 585 

Brelsford, W.M., 107 

Brigham, E.O., 26 

Brillinger, D.R., 330, 345 

Brockwell, P.J., 253, 328 

Burg, J.P., 211 


Carbone, R., 243 
Chatfield, Oe BED, 
Clayton, D.G., 360 
Cleveland, W.S., 115 
Cogburn, R., 183 
Cooley, J.W., 371 
Cox, D.R., 233 


Cybenko, G., 207 


Davis, H.T.., 183, 251, 272 


Davis, R.A., 253, 328 
DeMeersman, e212 
Draper, N.R., 399 
Durbin, J., 213 
Dzhaparidze, K., 197 


Engel, E.M.R., 156 
Ensor, K., 212, 259 


Feller, W., 173 
Fildes, R., 243 
Findley, DF, 275 
Fisher, R.A., 256 
Fishman, G.S., 295 
Fleeger, J.L., 591 
Friedlander, By 120 


Fuller, W.A., 74, 104, 118 


Gentleman, W.M.., 371 
Goodman, A.S., 578 
Granger, C.W.J., 156 


Gray, H.L., 216, 218, 254 


Grenander, Ue ra 43 


Hannan, E.J., 213, 222 
272, 330 


614 


, 299 


, 223, 224, 


AUTHOR INDEX 


Harms, P.G., 591 
Harrison, P.J., 246 
Hartley, H.O., 256 
Hibon, M., 243 


Isserlis, L., 166 


Jenkins, G.M., 113, 200, 202, 211, 
230, 232, 304, 343, 344, 585 

Jennrich, R.I., 397 

Jones, R.H., 107, 132, 197, 272 


Kailath, T., 120 
Kalman, R.E., 382 
Kedem, B., 53, 588 
Kohn, R., 114 


Levinson, N., 89, 120 
Lewandowski, R., 243 
Lewis, P.A.W., 578 
Ljung, G.M., 174 
Ljung, L., 120 


Makhoul, J., 205, 212 
Makridakis, S., 243 
Marsaglia, G., 578 
Masani, P., 247 
Meinhold, R.J., 383 
Mélard, G., 196 
Miller, J.M., 578 
Morf, M., 120 
Morris, J., 254, 587 
Morris, M.J., 156 


Newton, H.J., 107, 118, 129, 205, 
243, 252, 259, 591 

Nicholls, D.F., 272 

Nyquist, H., 272 


O’Neill, R., 196 
Owens, R.E., 591 


Pagano, M., 107, 118, 125, 129, 
135, 205, 252, 259 


615 


Parzen, E., 42, 45, 52, 167, 183, 189, 
194, 213, 218, 223, 243, 254, 259, 


299, 300, 340, 380, 589, 591 
Paulsen, J., 205 
Pierce, D.A., 174 


Priestley, M.B., 70, 74, 118, 155, 161, 
164, 189, 192, 204, 257, 345, 363 


Prothero, D.L., 232 
Pukkila, T., 272 


Quinn, B.G., 222, 223, 275 


Rahe, C.H., 591 
Rao, C.R., 362, 376 
Rissanen, J., 224 
Rosenblatt, M., 299 


Sande, G., 371 

Scheffé, H., 379 
Serfling, R.J., 387 
Shibata, R., 222, 251 
Siddiqui, M.M., 590 
Silverman, B.W., 45, 589 
Singleton, R.C., 374 
Singpurwalla, N.D., 383 
Smith, H., 399 

Stevens, C.F., 246 
Stewart, G.W., 208 
Stuart, A., 212 

Szego, G., 72, 143 


Tapia, R.A., 45 
Thompson, J.R., 45 
TiaoyG:Gs 213.2172 243 
Tjgstheim, D., 205 
Tong, H., 227 

Tsay, R.S., 213, 217, 292 
Tukey, J.W., 179, 371 


Ulrych, T.L., 211 
Van Ness, J.W., 193 


Wahba, G., 193 
Watts, D.G., 304, 585 


616 AUTHOR INDEX 


Welch, P.D., 298 Winkler, R., 243 
Whittle, P., 86, 197, 327 Woodroofe, M.B., 193 
Wilson, G., 96, 134, 135, 141 Woodward, W., 216, 218, 254 


SUBJECT INDEX 


See also Appendix C for an outline and description of the 
TIMESLAB commands, Appendix D for a listing of the data sets, 
and Appendix E for an index to the commands, theorems, macros, 
examples, and figures contained in the book. 


Absolute continuity, 73 

Absolute integrability, 365 

Absolute summability, 73, 117 

Akaike’s information criterion 
(AIC), 219-222, 224, 251, 
278, 293, 294, 298 

Aligning time series, 353 

Amplitude, 13 

Amplitude modulated process, 299 

Amplitude spectrum, 307, 310 

Antithetic variables, 295 

Approximate maximum likelihood 
estimators, 200 

Approximating filter coefficients, 320 

ARARMA modeling, 244 

Arctan2 function, 64 

ARIMA process, 112 

Arithmetic operations in 
TIMESLAB, 424 

ARMA spectral estimation, 253 

Asymptotically stationary, 158, 299 

Autocorrelation function, 71 

Autocorrelation method, 205 


617 


Autocovariance function 


Definition, 71 
Estimation of, 163 
Properties of, 72, 73 


Autocovariance method, 212 
Autoregressive (AR) process, 98-107 


Algorithms for, 123-129, 154, 
205-213 
Approximating processes by, 105 
Bivariate, 322, 327, 339, 352 
Burg estimators, 211, 273 
Estimation algorithms, 205-213 
Estimating peak frequencies, 
257-261 
Examples, 106 
Filter, 129, 290 
Least squares estimators, 212 
Partial autocorrelations of, 
106, 126 
Periodic, 107, 357 
Prediction of, 105, 130 
Properties, 101 
Simulating, 106, 128 


618 


Spectral estimation, 247-253 
Subset AR modeling, 227 
Test for stationarity, 125 
Yule-Walker estimators, 
205-210, 273 
Autoregressive-moving average 
(ARMA) process, 107-112 
Algorithms for, 129-134, 
140-142 
Approximate MLE’s, 200-202 
Bivariate, 328 
Exact MLE’s, 196-199 
Examples, 112 
Identifying, 214-240 
Likelihood function, 119 
Markovian representation, 132 
Partial autocorrelations of, 112 
Prediction of, 111, 129-131 
Properties, 109 
Simulating, 155 
Autoregressive spectral estimation, 
247-253 


Backforecasting, 201 
Backshift operator, 84 
Backward prediction coefficients, 123 
Banded matrix, 154 
Bandwidth, 44, 63, 178, 186 
Bandpass filter, 81-83, 145, 147, 155, 

157, 180, 181, 290, 366 
Bartlett’s formula, 167, 331 
Bartlett’s test, 170, 172, 267, 292 
Bayesian forecasting, 346 
Bauer’s algorithm, 135, 152 
Biased residual variances, 315 
Binary time series, 53 
Binomial distribution, 61 
Bivariate ARMA process, 328 
Bivariate autoregression, 322, 

327, 339, 352 
Bivariate CAT criterion, 340 
Bivariate difference equation, 325 
Bivariate Filter Theorem, 314 
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Bivariate general linear process, 329 

Bivariate spectral density estimation, 
330 

Bivariate white noise process, 311 

Boxcar function, 205 

Box-Jenkins modeling, 230-240 

Burg algorithm, 211, 273 


CAT criterion, 223, 247, 276 
Central limit theorem, 394 
Cesaro summability, 295 
Characteristic exponent, 189 
Characteristic function, 75, 377 
Characteristic polynomial, 40 
Characteristic value, 189 
Cholesky factors, 63, 96, 118, 358 
Circular shift operator, 305 
Clipped time series, 159 
Coherence spectrum, 307, 310, 
313; 355 
Companion matrix, 357 
Complex coherence spectrum, 307 
Conditional expectation, 375, 380 
Confidence bands for spectra, 
252, 287 
Confidence intervals 
For amplitude spectrum, 338 
For coherence, 338 
For f; 193, 252, 287 
For p, 162, 332 
For peak frequency, 259 
For phase spectrum, 338 
For R and p, 167, 332 
Conjugate pairs, 40 
Consistent estimator, 161 
Continuous function theorem, 
296, 387 
Convergence of random variables, 386 
Convolutions, 28, 117 
Correlogram, See Sample 
autocorrelation function 
Cospectral density, 307, 310 
Covariance generating function, 
79, 134, 140 
Covariance stationary, 70, 301 
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Cramer-Wold factorization, 134 Sales data, 232-240, 245, 
Cross-correlations, 8, 301 294, 587 
Cross-spectral analysis, 315 Sum of four cosines, 2, 4, 9, 12, 
Cross-spectral density, 306 18, 20, 23, 27, 565 
Cumulants, 166 Waves data, 290, 291, 593 
Cumulative periodogram, 22, 51, 172 Wolfer sunspot data, 69, 254, 
Cybenko’s theorem, 207 255, 292, 293, 587 
Delay process, 317, 356 
Data sets Designing filters, 83 
Airline Data, 1, 2, 4, 8, 9, 12,18, Difference equations, 38, 61, 67, 
20, 225:233 31, 35;.61,,.115, 201, 325 
203, 204, 219, 582 Differencing, 30, 59, 60, 65, 79, 
Artificial data, 2, 4, 8, 9, 12, 20, 81, 234 
23,31, 32; 292, 582 Dirichlet kernel, 82, 146, 157, 182 
Buffalo Snowfall, 63, 589 Discrete Fourier transform (DFT), 
Beveridge wheat price, 2, 6, 7, 8, 15, 26 


9, 12, 20, 23, 44, 45, 62,580 Disturbed periodicity, 257 
Critical radio frequencies, 194, DOS commands in TIMESLAB, 
195, 221, 222, 229, 248, 420, 434 
249, 294, 590 Dual process, 216 
Daily California births, 2, 6, 7, Duality of ARMA processes, 117 
9, 12, 18, 20, 23, 579 


Gas furnace data, 2, 3, 9, Editing files in TIMESLAB, 445-447 
333-335, 339-342, 344, Eigenvalues, 143 
345, 585 Ensemble, 76 
Lake Erie levels, 2, 4, 9, 12, Equivalent degrees of freedom, 
20; 23; 583 162, 291, 296 
Luteinizing hormone levels, Ergodic, 162 
294, 591 Errors in TIMESLAB, 428, 429 
Lynx data, 218, 219, 221, 227, Excess of high frequency, 18 
242, 253-255, 257, 260, Excess of low frequency, 18 
287, 288, 587 Exponential decay, 104 
Magnitude of a star, 194, 195, Exponential smoothing, 38, 57, 62, 66 
248, 249, 588 Extending data by inverse 
Mauna Loa CO? data, 589 differencing, 60 
Monthly sunspot data, 194, 
195, 248, 249, 590 Factoring covariance generating 
New York temperatures and functions, 134, 140, 204 
births, 592 Fast Fourier transform (FFT), 
Normal white noise data, 2, 3, 4, 24, 26, 371-374 


OML2 15; 20022523581 Fejér kernel, 169, 266 
Rainfall data, 2, 4, 9,12, 18,20, Filter, AR, 129, 290 

22, 23, 264, 265, 292, 581 Filter coefficients, 319 
Random walk series, 2, 8, 9,11, Filter design, 83 

12; 189209237583 Filter, optimal, 318 
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Filtered Poisson process, 254 

Filters, 78-86 

Filter theorem, 79, 314 

FIND mode in TIMESLAB, 419, 
440-442 

Fisher’s exact test, 256, 286, 299 

Forecasting, 35-42, 86-90, 118-134, 
230-240 

Forward prediction coefficients, 123 

Four-fifths rule, 192 

Fourier coefficients, 181, 365 

Fourier pairs, 24 

Fourier series, 181, 363-371 

Fourth-order cumulant, 166 

Fourth-order stationary, 166 

Frequency components, 11, 27, 75, 81 

Frequency response function, 83 

Frequency transfer function, 80, 
153, 354 

Fundamental frequency, 59 


Gain of a filter, 317 

Gain spectrum, 317, 356 

Gaussian time series, 70 

General linear model, 388 

General linear process, 117, 170 

Generalized least squares, 396 

Geometric series, 14, 26, 65 

Gibb’s phenomenon, 84 

GPAC array, 216 

Gram-Schmidt decomposition, 
63, 360 

Graphics in TIMESLAB, 435-445 


Hannan-Quinn modification of 
AIC, 222 

Harmonics, 19, 59 

Harmonic analysis, 11, 75 

Harmonic process, 92, 369 

Help in TIMESLAB, 430-431 

Hermitian matrix, 305 

High pass filter, 83 

Histograms, 42, 46, 62 

Homogeneous difference equation, 
40, 67 
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Horizon of a predictor, 86 
Hurwitz factor, 134 


Identifying ARMA models, 214-240 
Infinite memory prediction, 88 
Infinite order processes, 90, 101, 247 
Informative quantile function, 
42, 46, 63, 67 
Initial conditions for difference 
equations, 39 
Innovations, 88, 90 
Innovation variance, 88, 272 
Intervention analysis, 243 
Inverse autocorrelation, 115 
Inverse autocovariance, 115, 216 
Inverse differencing, 35, 62 
Inverse discrete transform (IDFT), 15 
Inverse partial autocorrelations, 216 
Inverse spectral density, 115 
Invertibility, 95 


Kalman filter algorithm, 118, 
131, 382 


Kernel density estimation, 43, 178 


Lag, 6 
Lag operator, 84 
Lag window, 181 
Lag window generator, 183, 292 
Lattice filter algorithm, 208 
Laurent expansion, 102 
Leakage, 179, 193 
Least squares estimation 

For AR processes, 212 

For regression models, 391 
Lebesgue-Stieltjes Integral, 73, 368 
Levinson’s algorithm, 88, 205, 327 
Likelihood function, 119 
Limit in mean square, 78, 386 
Linear filters, 78-86 
Linear trend, 30 
Log transform, 30 
Long memory time series, 4, 36 
Low pass filter, 83 
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MA spectral estimation, 253 
Macros in TIMESLAB, 447-465, 
599-601 
Markov process, 158 
Markovian representation of ARMA 
process, 132 
Martingale process, 92 
Matrices in TIMESLAB, 422, 473 
Matrix inversion lemma, 363 
Maximum likelihood estimation, 
120, 196-199 
Memory of a predictor, 86 
Method of moments estimators, 204 
Missing data, 52, 300 
Modified Cholesky decomposition, 
96, 118-131, 358-360 
Monthly means and variances, 33, 62 
Moving average (MA) process, 
78, 94-98 
Algorithms, 134-140, 152 
Bivariate, 314 
Nonidentifiability, 96, 153 
Estimation of, Se ARMA 
Processes 
Examples of, 98 
Partial autocorrelations of, 97 
Prediction of, 96, 131 
Properties, 80, 94 
Simulating, 86 
Moving average smoother, 56, 78, 81 
Multiple correlation, 10 
Multiplicative subset ARIMA 
process, 114, 230 
Multivariate normal distribution, 376 


Natural frequencies, 17, 170, 177 

Non-Gaussian time series, 118 

Nonidentifiability of MA processes, 
96, 153 

Nonlinear time series, 118, 155 

Nonparametric probability density 
estimation, 43 

Nonparametric spectral density 
estimation, 176-194 


Optimal bandwidth, 45, 192 
Optimal filter, 318 
Orthogonal contrasts, 11 
Orthogonal regression, 397 
Orthogonality of sinusoids, 13 


Padding with zeros, 24, 27-29 
Parametric spectral density 
estimation, 246-254 
Parseval’s relation, 366 
Parsimony, 230 
Partial autocorrelation function 
Definition, 88 
Estimation, 10, 210 
For AR process, 106, 126 
For ARMA process, 112 
For MA process, 96 
Inverse, 216 
Relation with prediction, 89 
Parzen kernel, 67, 84, 184 
p-Differentiable, 188, 296 
Peak frequency estimation, 257, 294 
Periodic autoregressive process, 
107, 357 
Periodogram, See Sample spectral 
density 
Permutation matrix, 87, 122, 
154, 157 
Phase of a complex number, 
13, 59, 63 
Phase spectrum, 307, 310, 335, 338 
Polynomials, 42, 125, 474 
Portmanteau test, 174, 268 
Positive definiteness, 72, 164, 358 
Power transformation, 30, 232 
Prediction, 35-42, 86-90, 118-134, 
230-240 
Prediction normal equations, 
88, 89, 123 
Prediction variance horizon (PVH), 
218, 275, 294 
Prewhitening, 349 
Printing in TIMESLAB, 433, 
439, 475 
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Purely nondeterministic time 
series, 88 


Q test, 174, 268 


Quadrature spectral density, 307-310 


Quasi-differencing, 244 


Random walk process, 90-92, 102, 
148, 153, 156, 291 
Recursive regression, 403 
Regression analysis, 31, 36, 56, 
61, 388-406 
Example, 399-402 
With AR errors, 240-243 
Residual variances, 10 
Resolving peaks, 194 


Sample autocorrelation function, 7 
Calculation of, 24, 28 
Definition, 6 
Examples, 8 
For sinusoid, 65 
Sampling properties, 166 

Sample partial autocorrelation 
function 
Calculation of, 210 
Definition, 10 
Examples, 11 
Sampling properties, 208 

Sample spectral density function, 
11-21 
Calculation of, 24 
Definition, 17 
Examples, 18 
Sampling properties, 168 

Sample spectral distribution 
function, 21 

Saturation, 251 

Saving files in TIMESLAB, 431 

Saving graphics screens in 
TIMESLAB, 442 

Sawtooth function, 54, 59 

Scatterplot, 6, 47 

Schur matrix, 125, 206, 252 

Schur test for stationarity, 125 


Search for periodicities, 160, 256 
Seasonal means and variances, 33, 66 
Seeds for random number generators 
434 
Serial correlation, 6 
Sidelobes, 82, 183 
Simple exponential smoothing, 39 
Simulating Data 
AR processes, 106, 128 
ARMA processes, 155 
MA processes, 86 
Random Walk, 92 
Simultaneous confidence intervals, 
379 
Sinusoid, 13, 65 
Sinusoidal decomposition, 15 
Smoothing, 33 
Spectral density function 
Definition, 73 
For AR process, 101 
For ARMA process, 109 
For MA process, 80 
Interpretation of, 75-76 
Spectral distribution function, 172 
Spectral representations, 72-76, 
367-371 
Spectral window, 181, 270 
Spectral window generator, 183 
Square root of a matrix, 358 
Squared coherence spectrum, 
307, 310 
Standardizing spectra, 19 
Starting values, 39, 196 
State space model, 246 
Stationary increments, 73 
Stepwise AR modeling, 227-229 
Stepwise ARMA modeling, 224-227 
Stepwise regression, 397 
Stochastic difference equation, 99 
Stochastic integral, 73 
Stochastic process, 370 
Strictly stationary, 71, 155 
Subset ARMA process, 112 
Sunspot cycle, 293 
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Superimposing plots in TIMESLAB, 
462 


Sweep algorithm, 361 


Testing 
Independence, 348 
Location of polynomial zeros, 
125 
White noise, 172-176 
Zero coherence, 351 
TIMESLAB, the program 
Commands, Appendix C 
Dictionary of commands, 477 
Editor, 445-447 
FIND, 440-442 
Graphics menu, 438 
Guided tour, 412-421 
Hardware and software needed, 
408 
Installing, 409 
Macros, 447-465, 599-601 
Outline of Commands, 467 


Scaling and labeling graphs, 443 
Tic mark labels, 444 
Toeplitz matrix, 71, 72, 87, 123, 157 
Tradeoff of bias and variance, 178 
Transfer function 
Estimation, 342 
Model, 353 
Transformations, 29, 231 
Truncation point, 182, 193 


Unbiased residual variances, 215 
Uncorrelated increments, 370 
Unit circle, 42 


White noise process, 76, 311 

White noise tests, 50, 172-176 
Wilson’s algorithms, 134-142 
Window spectral estimates, 176-193 
Wold decomposition, 89 


Yule-Walker equations, 101, 123, 204 
Yule-Walker estimators (YWE), 
205-210, 273 
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